<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<!--[if IE]><meta http-equiv="X-UA-Compatible" content="IE=edge"><![endif]-->
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<meta name="generator" content="Asciidoctor 1.5.8">
<title>Cloud Pipelines Concourse</title>
<link rel="stylesheet" href="css/spring.css">
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css">
</head>
<body class="article toc2 toc-left">
<div id="header">
<h1>Cloud Pipelines Concourse</h1>
<div id="toc" class="toc2">
<div id="toctitle">Table of Contents</div>
<ul class="sectlevel1">
<li><a href="#introduction">Introduction</a>
<ul class="sectlevel2">
<li><a href="#five-second-introduction">Five-second Introduction</a></li>
<li><a href="#five-minute-introduction">Five-minute Introduction</a></li>
<li><a href="#project-crawler">Project Crawler</a></li>
</ul>
</li>
<li><a href="#customizing-the-project">Customizing the Project</a>
<ul class="sectlevel2">
<li><a href="#customization-overriding-project-setup">Overriding Project Setup</a></li>
<li><a href="#customization-overriding-pipelines">Overriding Pipelines</a></li>
</ul>
</li>
<li><a href="#step-by-step-cloud-foundry-migration">Step-by-step Cloud Foundry Migration</a>
<ul class="sectlevel2">
<li><a href="#preview">Preview</a></li>
<li><a href="#introduction-2">Introduction</a></li>
<li><a href="#sample-applicationinitial-state">Sample Application&#8201;&#8212;&#8201;Initial State</a></li>
<li><a href="#sample-applicationend-state">Sample Application&#8201;&#8212;&#8201;End State</a></li>
<li><a href="#tutorialtoolset">Tutorial&#8201;&#8212;&#8201;Toolset</a></li>
<li><a href="#tutorialoverview">Tutorial&#8201;&#8212;&#8201;Overview</a></li>
<li><a href="#tutorialstep-by-step">Tutorial&#8201;&#8212;&#8201;Step-by-step</a></li>
<li><a href="#conclusion">Conclusion</a></li>
</ul>
</li>
<li><a href="#setup-examples">Setup examples</a></li>
<li><a href="#building-the-project">Building the Project</a>
<ul class="sectlevel2">
<li><a href="#building-project-setup">Project Setup</a></li>
<li><a href="#building-prerequisites">Prerequisites</a></li>
<li><a href="#building-bats-submodules">Bats Submodules</a></li>
<li><a href="#building-build-and-test">Build and test</a></li>
<li><a href="#building-generate-documentation">Generate Documentation</a></li>
</ul>
</li>
<li><a href="#building-ci-worker-prerequisites">CI Server Worker Prerequisites</a></li>
<li><a href="#concourse-faq">Concourse FAQ</a></li>
</ul>
</div>
</div>
<div id="content">
<div id="preamble">
<div class="sectionbody">
<div class="paragraph">
<p>This project contains setup for Concourse that creates jobs and pipelines
for your projects. The pipelines and jobs use the scripts defined in
<a href="https://github.com/CloudPipelines/scripts">Cloud Pipelines Scripts</a> repo.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="introduction"><a class="link" href="#introduction">Introduction</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>This section describes how Concourse works with Cloud Pipelines.</p>
</div>
<div class="admonitionblock important">
<table>
<tr>
<td class="icon">
<i class="fa icon-important" title="Important"></i>
</td>
<td class="content">
You do not need to use all the pieces of Cloud Pipelines. You
can (and should) gradually migrate your applications to use those pieces of
Cloud Pipelines that you think best suit your needs.
</td>
</tr>
</table>
</div>
<div class="sect2">
<h3 id="five-second-introduction"><a class="link" href="#five-second-introduction">Five-second Introduction</a></h3>
<div class="paragraph">
<p>Cloud Pipelines Concourse provides setup for Concourse that creates jobs and pipelines.
The pipelines and jobs use the scripts defined in
<a href="https://github.com/CloudPipelines/scripts">Cloud Pipelines Scripts</a> repo.</p>
</div>
</div>
<div class="sect2">
<h3 id="five-minute-introduction"><a class="link" href="#five-minute-introduction">Five-minute Introduction</a></h3>
<div class="paragraph">
<p>In these sections you will learn how exactly Cloud Pipelines Concourse integrates
with <a href="https://github.com/CloudPipelines/scripts">Cloud Pipelines Scripts</a> and how
you can setup deployment pipelines for each project.</p>
</div>
<div class="sect3">
<h4 id="how-to-use-it"><a class="link" href="#how-to-use-it">How to Use It</a></h4>
<div class="paragraph">
<p>The suggested approach is to use the <a href="#project-crawler">Project Crawler</a> approach, where
we scan your organization for projects and create deployment pipeline for each.</p>
</div>
<div class="paragraph">
<p>Another approach is to add the contents of each project and alter it to suit your needs.</p>
</div>
</div>
<div class="sect3">
<h4 id="how-it-works"><a class="link" href="#how-it-works">How It Works</a></h4>
<div class="paragraph">
<p>As the following image shows, Cloud Pipelines contains logic to generate a
pipeline and the runtime to execute pipeline steps.</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/intro/how.png" alt="how">
</div>
<div class="title">Figure 1. How Cloud Pipelines works</div>
</div>
<div class="paragraph">
<p>Once a pipeline is created the jobs are ran, they clone or download Cloud Pipelines
code to run each step. Those steps run functions that are
defined in <a href="https://github.com/CloudPipelines/scripts">Cloud Pipelines Scripts</a>.</p>
</div>
<div class="paragraph">
<p>Cloud Pipelines performs steps to guess what kind of a project your
repository is (e.g. JVM or PHP) and what framework it uses (Maven or Gradle), and it
can deploy your application to a cloud (e.g. Cloud Foundry or Kubernetes)</p>
</div>
<div class="paragraph">
<p>Cloud Pipelines Concourse contains bash scripts that are required at runtime of
execution of a Concourse pipeline. If you want to read its documentation, it&#8217;s
available under <a href="https://github.com/{org}/{repo}/blob/{branch}/src/tasks/README.adoc"><code>src/tasks/README.adoc</code></a> file of Cloud Pipelines repository.</p>
</div>
<div class="paragraph">
<p>You can <a href="BASH_SCRIPTS.html">click here to go the separate subpage containing that documentation</a>.</p>
</div>
</div>
</div>
<div class="sect2">
<h3 id="project-crawler"><a class="link" href="#project-crawler">Project Crawler</a></h3>
<div class="paragraph">
<p>Cloud Pipelines Concourse comes with a <code>crawler.groovy</code> file that allows to
go through the repositories in an organization in an SCM server (e.g. Github, Gitlab, Bitbucket)
and for each repo it will create a pipeline in Concourse.</p>
</div>
<div class="paragraph">
<p>We use the <a href="https://github.com/CloudPipelines/project-crawler">Project Crawler</a>
library, which can:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Fetch all projects for a given organization.</p>
</li>
<li>
<p>Fetch contents of a file for a given repository.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For your convenience we&#8217;re providing you with a script under <code>src/pipeline/set_pipeline_crawler.sh</code>
that:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Verifies if Groovy is installed. If not, fetches it and unpacks to <code>build/groovy</code></p>
</li>
<li>
<p>Defines all the required environment variables</p>
</li>
<li>
<p>Runs the <code>crawler.groovy</code> script</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The following diagram depicts this situation:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="crawler.png" alt="crawler" width="1053" height="590">
</div>
</div>
<div class="paragraph">
<p>Project Crawler supports repositories stored at Github, Gitlab, and Bitbucket.
You can also register your own implementation. See the
<a href="https://github.com/CloudPipelines/project-crawler">Project Crawler</a> repository for more information.</p>
</div>
<div class="paragraph">
<p>Below you can find the environment variables required by the Bash script</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash"># root URL of the SCM server's API
export CRAWLER_ROOTURL="${CRAWLER_ROOTURL:-https://github.com}"
# username to connect to the API
export CRAWLER_USERNAME="${CRAWLER_USERNAME:-}"
# password to connect to the API
export CRAWLER_PASSWORD="${CRAWLER_PASSWORD:-}"
# token to connect to the API
export CRAWLER_TOKEN="${CRAWLER_TOKEN:-}"
# pattern to exclude certain projects
export CRAWLER_REPOPROJECTSEXCLUDEPATTERN="${CRAWLER_REPOPROJECTSEXCLUDEPATTERN:-}"
# type of SCM repo server (GITHUB, GITLAB, BITBUCKET, OTHER). Can be resolved from URL
export CRAWLER_REPOTYPE="${CRAWLER_REPOTYPE:-}"
# name of the pipeline descriptor
export CRAWLER_ORG="${CRAWLER_ORG:-}"
# organization / project name
export CRAWLER_PIPELINEDESCRIPTOR="${CRAWLER_PIPELINEDESCRIPTOR:-}"
# alias to be used by FLY CLI to set the pipeline
export CRAWLER_ALIAS="${CRAWLER_ALIAS:-}"
# credentials file to be used by FLY CLI to set the pipeline
export CRAWLER_CREDENTIALS="${CRAWLER_CREDENTIALS:-}"</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Here you can find the description of the Groovy script</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">	The arguments for this script will be taken in the order:
	Arguments, System Properties, Environment Variables

	Arguments:
	0 - rootUrl - root URL of the SCM server's API
	1 - username - username to connect to the API
	2 - password - password to connect to the API
	3 - token - token to connect to the API
	4 - repoProjectsExcludePattern - pattern to exclude certain projects
	5 - repoType - type of SCM repo server (GITHUB, GITLAB, BITBUCKET, OTHER)
	6 - org - organization / project name
	7 - pipelineDescriptor - name of the pipeline descriptor
	8 - alias - alias to be used by FLY CLI to set the pipeline
	9 - credentials - credentials file to be used by FLY CLI to set the pipeline

	System props with the name equal to arguments (e.g. rootUrl)
	Environment variables are upper case arguments with CRAWLER_ prefix (e.g. CRAWLER_ROOTURL)

	E.g. of calling the script

	#!/bin/bash

	set -o errexit
	set -o errtrace
	set -o pipefail

	export CRAWLER_ROOTURL="https://github.com"
	export CRAWLER_USERNAME="username"
	export CRAWLER_PASSWORD="password"
	export CRAWLER_TOKEN=""
	export CRAWLER_REPOPROJECTSEXCLUDEPATTERN=""
	export CRAWLER_REPOTYPE=""
	export CRAWLER_ORG="my-org"
	export CRAWLER_PIPELINEDESCRIPTOR="my-pipelines.yml"
	export CRAWLER_ALIAS="alias"
	export CRAWLER_CREDENTIALS="credentials-cf.yml"

	./set_pipeline_crawler.sh</code></pre>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="customizing-the-project"><a class="link" href="#customizing-the-project">Customizing the Project</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Cloud Pipelines offers a number of ways to customize a Pipelines project:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="#customization-overriding-project-setup">Overriding Project Setup</a></p>
</li>
<li>
<p><a href="#customization-overriding-pipelines">Overriding Pipelines</a></p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="customization-overriding-project-setup"><a class="link" href="#customization-overriding-project-setup">Overriding Project Setup</a></h3>
<div class="paragraph">
<p>If you want to customize the Cloud Pipelines build, you can update the contents
of the <code>gradle/custom.gradle</code> build script. That way your customizations will not
interfere with the changes in the main part of the code, thus there should be
no merge conflicts when pulling the changes from Cloud Pipeline repositories.</p>
</div>
</div>
<div class="sect2">
<h3 id="customization-overriding-pipelines"><a class="link" href="#customization-overriding-pipelines">Overriding Pipelines</a></h3>
<div class="paragraph">
<p>Currently, the best way to extend the Concourse is to make
a copy of the Concourse pipeline <code>yaml</code> files and modify it.</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="step-by-step-cloud-foundry-migration"><a class="link" href="#step-by-step-cloud-foundry-migration">Step-by-step Cloud Foundry Migration</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>This section details how to migrate applications such that they become compatible with  Cloud Pipelines.</p>
</div>
<div class="sect2">
<h3 id="preview"><a class="link" href="#preview">Preview</a></h3>
<div class="paragraph">
<p><a href="https://docs.google.com/presentation/d/e/2PACX-1vSsEHn8cJfz8oWIwwUhdULt7nZzz3bBLK7OqM8UInkZ0LbQBCpPdhMoxsYGPe_90h9OvCu7dFlAimMJ/pub?start=false&amp;loop=false&amp;delayms=3000">Click here</a> to
check out the slides by <a href="https://twitter.com/ciberkleid">Cora Iberkleid</a> where she
migrates a set of applications to be compliant with Cloud Pipelines.</p>
</div>
</div>
<div class="sect2">
<h3 id="introduction-2"><a class="link" href="#introduction-2">Introduction</a></h3>
<div class="paragraph">
<p>This tutorial covers refactoring applications to be compatible with, and take advantage of, Cloud Pipelines.</p>
</div>
<div class="paragraph">
<p>As an example, we use a simple three-tier application, shown in the following image:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/use_case_logical.png" alt="use case logical">
</div>
<div class="title">Figure 2. Use Case - Logical View</div>
</div>
<div class="paragraph">
<p>At the end of this tutorial, you will be able to quickly create a Concourse pipeline for each application and run successfully through a full lifecycle, from source code commit to production deployment, following the lifecycle stages for testing and deployment recommended by Cloud Pipelines. You will be able to improve application code bases with organized test coverage, a contract-based API, and a versioned database schema, letting Cloud Pipelines carry out stubbed testing and ensure backward compatibility for API and database schema changes.</p>
</div>
</div>
<div class="sect2">
<h3 id="sample-applicationinitial-state"><a class="link" href="#sample-applicationinitial-state">Sample Application&#8201;&#8212;&#8201;Initial State</a></h3>
<div class="paragraph">
<p>The sample application is implemented by using Spring Boot applications for the UI and service tiers and MySQL for the database.</p>
</div>
<div class="paragraph">
<p>The apps are built with Maven and manually pushed to Cloud Foundry. They leverage the three Pivotal Spring Cloud Services: Config Server, Service Discovery, and Circuit Breaker Dashboard. We use Rabbit to propagate Config Server refresh triggers.</p>
</div>
<div class="paragraph">
<p>The source code for the two Spring Boot applications is stored on GitHub, as is the backing repo for Config Server.</p>
</div>
<div class="paragraph">
<p>The following image shows an implementation view of the applications and their ancillary services:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/use_case_implementation.png" alt="use case implementation">
</div>
<div class="title">Figure 3. Use Case - Implementation</div>
</div>
</div>
<div class="sect2">
<h3 id="sample-applicationend-state"><a class="link" href="#sample-applicationend-state">Sample Application&#8201;&#8212;&#8201;End State</a></h3>
<div class="paragraph">
<p>Throughout this tutorial, we add Concourse and JFrog Bintray to manage the application lifecycle.</p>
</div>
<div class="paragraph">
<p>We also refactor the applications so that they become compatible with Cloud Pipelines requirements and recommendations, including adding and organizing tests and introducing database versioning by using Flyway and introducing API contracts by using Spring Cloud Contract.</p>
</div>
</div>
<div class="sect2">
<h3 id="tutorialtoolset"><a class="link" href="#tutorialtoolset">Tutorial&#8201;&#8212;&#8201;Toolset</a></h3>
<div class="paragraph">
<p>Throughout this tutorial, we use the following tools:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><strong>GitHub</strong>: Sample application source code and configuration repositories, a sample stubrunner application repository, and the Cloud Pipelines code base, including the following:</p>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/ciberkleid/greeting-ui"><code>greeting-ui</code></a></p>
</li>
<li>
<p><a href="https://github.com/ciberkleid/fortune-service"><code>fortune-service</code></a></p>
</li>
<li>
<p><a href="https://github.com/ciberkleid/app-config"><code>app-config</code></a></p>
</li>
<li>
<p><a href="https://github.com/spring-cloud-samples/cloudfoundry-stub-runner-boot"><code>cloudfoundry-stub-runner-boot</code></a></p>
</li>
<li>
<p><a href="https://github.com/CloudPipelines/concourse"><code>Cloud Pipelines Concourse</code></a></p>
</li>
<li>
<p><a href="https://github.com/CloudPipelines/scripts"><code>Cloud Pipelines Scripts</code></a></p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>Pivotal Web Services</strong>: Publicly hosted Cloud Foundry offering <a href="https://run.pivotal.io">free trial accounts</a> and including MySQL, Rabbit, and Pivotal Spring Cloud Services in the Marketplace</p>
</li>
<li>
<p><strong>Concourse</strong></p>
</li>
<li>
<p><strong>JFrog Bintray</strong>: Publicly hosted Maven repository offering free <a href="https://bintray.com/signup/oss">OSS accounts</a></p>
</li>
<li>
<p><strong>Client Tools</strong>: On your local machine, you need an IDE as well as the mvn, git, cf, and fly (Concourse) CLIs</p>
</li>
</ul>
</div>
</div>
<div class="sect2">
<h3 id="tutorialoverview"><a class="link" href="#tutorialoverview">Tutorial&#8201;&#8212;&#8201;Overview</a></h3>
<div class="paragraph">
<p>We separate the migration steps into three stages:</p>
</div>
<div class="olist arabic">
<ol class="arabic">
<li>
<p><strong>Scaffolding</strong></p>
<div class="ulist">
<ul>
<li>
<p>Minimal refactoring to be compatible with basic Cloud Pipelines requirements.</p>
</li>
<li>
<p>At the end of this stage, each application has a corresponding pipeline on Concourse. The pipelines successfully build the applications, store the artifacts in Bintray, tag the GitHub repositories, and deploy the applications to the Test, Stage, and Prod spaces in Cloud Foundry.</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>Tests</strong></p>
<div class="ulist">
<ul>
<li>
<p>Add and organize tests to be compatible with Cloud Pipelines recommendations.</p>
</li>
<li>
<p>Incorporate flyway for database schema versioning and initial data loading.</p>
</li>
<li>
<p>At the end of this stage, the pipelines trigger unit and integration tests during the Build stage, smoke tests in the Test environment, and end-to-end tests in the Stage environment. The pipelines also ensure backward compatibility for the database, such that you can safely roll back the backend service application, even after the database schema has been updated.</p>
</li>
</ul>
</div>
</li>
<li>
<p><strong>Contracts</strong></p>
<div class="ulist">
<ul>
<li>
<p>Incorporate Spring Cloud Contract to define the API between the UI and service apps and auto-generate tests and stubs.</p>
</li>
<li>
<p>At the end of this stage, the pipelines catch breaking API changes during the Build stage and ensure backward compatibility for the API, such that you can safely roll back the backend service (producer) app, even after an API change.</p>
</li>
</ul>
</div>
</li>
</ol>
</div>
</div>
<div class="sect2">
<h3 id="tutorialstep-by-step"><a class="link" href="#tutorialstep-by-step">Tutorial&#8201;&#8212;&#8201;Step-by-step</a></h3>
<div class="paragraph">
<p>The remainder of this chapter is the actual tutorial, which consists of a preparation stage and three main stages:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="#tutorial-prep">Prep: Before you begin</a></p>
</li>
<li>
<p><a href="#tutorial-stage-one">Stage One: Scaffolding</a></p>
</li>
<li>
<p><a href="#tutorial-stage-two">Stage Two: Tests</a></p>
</li>
<li>
<p><a href="#tutorial-stage-three">Stage Three: Contracts</a></p>
</li>
</ul>
</div>
<div class="sect3">
<h4 id="tutorial-prep"><a class="link" href="#tutorial-prep">Prep: Before you begin</a></h4>
<div class="paragraph">
<p>If you want to simply review the migration steps explained below, you can look at the various branches in the <a href="https://github.com/ciberkleid/greeting-ui">greeting-ui</a> and <a href="https://github.com/ciberkleid/fortune-service">fortune-service</a> repositories. A branch represents the end-state of each stage, as the following image shows:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/github_branches.png" alt="github branches">
</div>
<div class="title">Figure 4. GitHub Branches</div>
</div>
<div class="paragraph">
<p>If you want to use this tutorial as a hands-on lab, fork each of the following repositories:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/ciberkleid/greeting-ui">greeting-ui</a></p>
</li>
<li>
<p><a href="https://github.com/ciberkleid/fortune-service">fortune-service</a></p>
</li>
<li>
<p><a href="https://github.com/ciberkleid/app-config">app-config</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Then create a new directory on your local machine. You may name it anything you like. We refer to it as <code>$SCP_HOME</code> throughout this tutorial.</p>
</div>
<div class="paragraph">
<p>In <code>$SCP_HOME</code>, clone your forks of <code>greeting-ui</code> and <code>fortune-service</code>, as well as the following two repositories:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="https://github.com/spring-cloud-samples/cloudfoundry-stub-runner-boot">cloudfoundry-stub-runner-boot</a></p>
</li>
<li>
<p><a href="https://github.com/CloudPipelines/concourse">Cloud Pipelines Concourse</a></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Finally, create a directory called <code>$SCP_HOME/credentials</code>. Leave it empty for now.</p>
</div>
</div>
<div class="sect3">
<h4 id="tutorial-stage-one"><a class="link" href="#tutorial-stage-one">Stage One: Scaffolding</a></h4>
<div class="paragraph">
<p>In this stage, we make minimal changes to satisfy basic Cloud Pipelines requirements so that the applications can run through the entire pipeline without error. We make &#8220;scaffolding&#8221; changes only&#8201;&#8212;&#8201;no code changes.</p>
</div>
<div class="admonitionblock important">
<table>
<tr>
<td class="icon">
<i class="fa icon-important" title="Important"></i>
</td>
<td class="content">
You must complete the steps in this stage for both <code>greeting-ui</code> and <code>fortune-service</code>.
</td>
</tr>
</table>
</div>
<div class="sect4">
<h5 id="1-1-create-github-branches"><a class="link" href="#1-1-create-github-branches">1.1 Create GitHub Branches</a></h5>
<div class="paragraph">
<p>Create branches in GitHub by using the following git commands:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">git branch version
git checkout -b cloud-pipelines</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>The <code>version</code> branch is required to exist, though it can be created as an empty branch. It is used by Spring Coud Pipelines to generate a version number for each new pipeline execution.</p>
</div>
<div class="paragraph">
<p>The <code>cloud-pipelines</code> branch is optional and can be named anything you wish. The intention is for you to use it as a working branch for the changes suggested in this tutorial (hence, you should both create it and check it out).</p>
</div>
</div>
<div class="sect4">
<h5 id="1-2-add-maven-wrapper"><a class="link" href="#1-2-add-maven-wrapper">1.2 Add Maven Wrapper</a></h5>
<div class="paragraph">
<p>This step covers how to add the Maven wrapper (which lets your users build without having Maven on the path). To add the Maven wrapper, run the following command:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">mvn -N io.takari:maven:wrapper</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>This commands adds four files to a project:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code>.
├── mvnw
├── mvnw.cmd
└── .mvn
    └── wrapper
        ├── maven-wrapper.jar
        └── maven-wrapper.properties</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Make sure all four files are tracked by Git. One way to do so is to add the following lines to the <code>.gitignore</code> file:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code>#Exceptions
!/mvnw
!/mvnw.cmd
!/.mvn/wrapper/maven-wrapper.jar
!/.mvn/wrapper/maven-wrapper.properties</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="1-3-create-the-bintray-maven-repository-package"><a class="link" href="#1-3-create-the-bintray-maven-repository-package">1.3 Create the Bintray Maven Repository Package</a></h5>
<div class="paragraph">
<p>We use Bintray as the Maven repository. Bintray requires that a package exist before any application artifacts can be uploaded.</p>
</div>
<div class="paragraph">
<p>Log into the Bintray UI and create the packages as follows (you can use the &#8220;Import from GitHub&#8221; option to create these):</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/bintray_packages.png" alt="bintray packages">
</div>
<div class="title">Figure 5. Bintray Packages</div>
</div>
</div>
<div class="sect4">
<h5 id="1-4-configure-distribution-management-by-using-the-bintray-maven-repository"><a class="link" href="#1-4-configure-distribution-management-by-using-the-bintray-maven-repository">1.4 Configure Distribution Management by Using the Bintray Maven Repository</a></h5>
<div class="admonitionblock important">
<table>
<tr>
<td class="icon">
<i class="fa icon-important" title="Important"></i>
</td>
<td class="content">
You must do this step for both application repositories.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Edit the application <code>pom.xml</code> files. Make sure that the Bintray URLs match the URLs of the corresponding packages created in the previous step. The values you use should differ from the following example in that they should point to your repository:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">&lt;properties&gt;
...
&lt;distribution.management.release.id&gt;bintray&lt;/distribution.management.release.id&gt;
&lt;distribution.management.release.url&gt;https://api.bintray.com/maven/ciberkleid/maven-repo/fortune-service&lt;/distribution.management.release.url&gt;
&lt;/properties&gt;

...

&lt;distributionManagement&gt;
&lt;repository&gt;
&lt;id&gt;${distribution.management.release.id}&lt;/id&gt;
&lt;url&gt;${distribution.management.release.url}&lt;/url&gt;
&lt;/repository&gt;
&lt;/distributionManagement&gt;</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Though not required by Cloud Pipelines, it makes sense to also configure your local maven settings with the credentials to your Bintray maven repository. To do so, edit your maven settings file (usually <code>~/.m2/settings.xml</code>). If the file does not exist, create it.</p>
</div>
<div class="paragraph">
<p>Note that the <code>id</code> must match the <code>id</code> specified in the previous step. Also, make sure to use your username and API token (not your account password) instead of the sample values shown in the following example:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">&lt;?xml version="1.0" encoding="UTF-8"?&gt;
&lt;settings&gt;
  &lt;servers&gt;
    &lt;server&gt;
      &lt;id&gt;bintray&lt;/id&gt;
      &lt;username&gt;ciberkleid&lt;/username&gt;
      &lt;password&gt;my-super-secret-api-token&lt;/password&gt;
   &lt;/server&gt;
 &lt;/servers&gt;
&lt;/settings&gt;</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="1-5-push-changes-to-github"><a class="link" href="#1-5-push-changes-to-github">1.5 Push Changes to GitHub</a></h5>
<div class="paragraph">
<p>Push the changes you made in the preceding step to GitHub. You should be pushing the following to each of the two application repositories:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Four new Maven wrapper files</p>
</li>
<li>
<p>A modified <code>.gitignore</code> file</p>
</li>
<li>
<p>A modified <code>pom.xml</code> file</p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="1-6-add-a-cloud-pipelines-credentials-file"><a class="link" href="#1-6-add-a-cloud-pipelines-credentials-file">1.6 Add a Cloud Pipelines Credentials File</a></h5>
<div class="paragraph">
<p>In <code>$SCP_HOME/credentials</code>, make two copies of the <code>$SCP_HOME/concourse/src/pipeline/credentials-sample-cf.yml</code> file. Rename them as <code>credentials-fortune-service.yml</code> and <code>credentials-greeting-ui.yml</code>.</p>
</div>
<div class="admonitionblock caution">
<table>
<tr>
<td class="icon">
<i class="fa icon-caution" title="Caution"></i>
</td>
<td class="content">
These files will contain credentials to your GitHub repository, your Bintray repository, and your Cloud Foundry foundation. Hence, we opt to put them in a separate directory. You may choose to store these files in a private Git repository, but do not push them to a public repository.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>Edit the Git properties of each credentials file. Make sure to replace the sample values shown in our example. For <code>concourse-scripts-branch</code>, you can use a fixed release (use v1.0.0.M8 or later for Cloud Foundry). Leave the other values as they are. We update those in later steps. The following listing shows a credentials file:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">app-url: git@github.com:ciberkleid/fortune-service.git
app-branch: cloud-pipelines
scripts-url: https://github.com/CloudPipelines/scripts.git
scripts-branch: master
concourse-scripts-url: https://github.com/CloudPipelines/concourse.git
concourse-scripts-branch: master
build-options: ""

github-private-key: |
  -----BEGIN RSA PRIVATE KEY-----
  MIIJKQIBAAKCAgEAvwkL97vBllOSE39Wa5ppczT1cr5Blmkhadfoa1Va2/IBVyvk
  NJ9PqoTI+BahF2EgzweyiDSvKsstlTsG7QgiM9So8Voi2PlDOrXL6uOfCuAS/G8X
  ...
  -----END RSA PRIVATE KEY-----
git-email: ciberkleid@pivotal.io
git-name: Cora Iberkleid</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Edit the Maven repository properties of each credentials file. Make sure to replace the sample values shown in our example. Bintray requires separate URLs for uploads and downloads. If you use a different artifact repository, such as Artifactory or Nexus, and the repository URL is the same for uploads and downloads, you do not need to set <code>repo-with-binaries-for-upload</code>. The following listing shows the values to add or edit in your credentials file:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">m2-settings-repo-id: bintray
m2-settings-repo-username: ciberkleid
m2-settings-repo-password: my-super-secret-api-token

repo-with-binaries: https://ciberkleid:my-super-secret-api-token@dl.bintray.com/ciberkleid/maven-repo

repo-with-binaries-for-upload: https://api.bintray.com/maven/ciberkleid/maven-repo/fortune-service</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="1-7-set-the-concourse-pipeline"><a class="link" href="#1-7-set-the-concourse-pipeline">1.7 Set the Concourse Pipeline</a></h5>
<div class="paragraph">
<p>At this point, all of the build jobs, which run on Concourse workers, should succeed.</p>
</div>
<div class="paragraph">
<p>To verify this, log in to your Concourse target and set the Concourse pipelines. Update the target name in the following example:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash"># Set greeting-ui pipeline
fly -t myTarget set-pipeline -p greeting-ui -c "${SCP_HOME}/concourse/src/pipeline/pipeline.yml" -l "${SCP_HOME}/credentials/credentials-greeting-ui.yml" -n

# Set fortune-service pipeline
fly -t myTarget set-pipeline -p fortune-service -c "${SCP_HOME}/concourse/src/pipeline/pipeline.yml" -l "${SCP_HOME}/credentials/credentials-fortune-service.yml" -n</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Log into the Concourse UI and un-pause the pipelines. Start each pipeline. You should see that the build jobs all succeed, similar to the following image:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/concourse_build_success.png" alt="concourse build success">
</div>
<div class="title">Figure 6. Build Success</div>
</div>
<div class="paragraph">
<p>In addition, you should see a new <code>dev/&lt;app_name&gt;/&lt;version_number&gt;</code> tag in each GitHub repository and see the app jars uploaded into Bintray.</p>
</div>
<div class="paragraph">
<p>The test, stage, and prod jobs fail, because we have not yet added scaffolding for deployment to Cloud Foundry. We do that next.</p>
</div>
</div>
<div class="sect4">
<h5 id="1-8-add-cloud-foundry-manifest"><a class="link" href="#1-8-add-cloud-foundry-manifest">1.8 Add Cloud Foundry manifest</a></h5>
<div class="paragraph">
<p>If you are deploying to Cloud Foundry, you may already be routinely including manifest files with your applications. Our sample applications did not have manifest files, so we add them now.</p>
</div>
<div class="paragraph">
<p>In the <code>greeting-ui</code> repository, create a <code>manifest.yml</code> file as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">---
applications:
- name: greeting-ui
  timeout: 120
  services:
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>In the <code>fortune-service</code> repository, create a <code>manifest.yml</code> file as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">---
applications:
- name: fortune-service
  timeout: 120
  services:
  - fortune-db
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>The <code>TRUST_CERTS</code> variable is used by the Pivotal Spring Cloud Services (Config Server, Service Registry, and Circuit Breaker Dashboard), which we use in this example. The value specified in the preceding example assumes deployment to Pivotal Web Services. Update it accordingly if you are deploying to a different Cloud Foundry foundation, or you can leave it out altogether if you are replacing the Pivotal Spring Cloud Services with alternative implementations (for example, deploying the services as applications and exposing them as user-provided services).</p>
</div>
<div class="paragraph">
<p>If you wish, you can add additional values to the manifest files&#8201;&#8212;&#8201;for example, if additional values are useful for any manual deployment you may still want to do or if you need additional values in your Cloud Pipelines deployment. For example, the following file could be an alternative <code>manifest.yml</code> for <code>fortune-service</code>:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">---
applications:
- name: fortune-service
  timeout: 120
  instances: 3
  memory: 1024M
  buildpack: https://github.com/cloudfoundry/java-buildpack.git
  random-route: true
  path: ./target/fortune-service-0.0.1-SNAPSHOT.jar
  services:
  - fortune-db
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    SPRING_PROFILES_ACTIVE: someProfile
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Note that Cloud Pipelines ignores <code>random-route</code> and <code>path</code>. <code>instances</code> is honored in stage and prod but is overridden with a value of 1 for test.</p>
</div>
</div>
<div class="sect4">
<h5 id="1-9-add-the-cloud-pipelines-manifest"><a class="link" href="#1-9-add-the-cloud-pipelines-manifest">1.9 Add the Cloud Pipelines Manifest</a></h5>
<div class="paragraph">
<p>The Cloud Foundry manifest created in the previous step includes the logical names of the services to which the applications should be bound, but it does not describe how the services can be provisioned. Hence, we add a second manifest file so that Cloud Pipelines can provision the services.</p>
</div>
<div class="paragraph">
<p>Add a file called <code>cloud-pipelines.yml</code> to each application and include the same list of services as in the corresponding <code>manifest.yml</code>. Add the necessary details such that Cloud Pipelines can construct a <code>cf create-service</code> command.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
The `type: broker' parameter in the next example instructs Cloud Pipelines to provision a service by using `cf create-service'. Other service types are also supported: cups, syslog, route, app, and stubrunner.
</td>
</tr>
</table>
</div>
<div class="paragraph">
<p>More specifically, for <code>greeting-ui</code>, create an <code>cloud-pipelines.yml</code> file with the following content:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">test:
  services:
    - name: config-server
      type: broker
      broker: p-config-server
      plan: standard
      params:
        git:
          uri: https://github.com/ciberkleid/app-config
      useExisting: true
    - name: cloud-bus
      type: broker
      broker: cloudamqp
      plan: lemur
      useExisting: true
    - name: service-registry
      type: broker
      broker: p-service-registry
      plan: standard
      useExisting: true
    - name: circuit-breaker-dashboard
      type: broker
      broker: p-circuit-breaker-dashboard
      plan: standard
      useExisting: true</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>The <code>cloud-pipelines.yml</code> file for <code>fortune-service</code> is similar, with the addition of the <code>fortune-db</code> service, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">test:
  # list of required services
  services:
    - name: fortune-db
      type: broker
      broker: cleardb
      plan: spark
      useExisting: true
    - name: config-server
      type: broker
      broker: p-config-server
      plan: standard
      params:
        git:
          uri: https://github.com/ciberkleid/app-config
      useExisting: true
    - name: cloud-bus
      type: broker
      broker: cloudamqp
      plan: lemur
      useExisting: true
    - name: service-registry
      type: broker
      broker: p-service-registry
      plan: standard
      useExisting: true
    - name: circuit-breaker-dashboard
      type: broker
      broker: p-circuit-breaker-dashboard
      plan: standard
      useExisting: true</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>The values in the preceding two examples assume deployment to Pivotal Web Services. If you are deploying to a different Cloud Foundry foundation, update the values accordingly. Also, make sure to replace the <code>config-server</code> URI with the address of your fork of the <a href="https://github.com/ciberkleid/app-config"><code>app-config</code></a> repository.</p>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
Notice the <code>useExisting: true</code> parameter in the preceding example. By default, Cloud Pipelines deletes and re-creates services in the <code>test</code> space. To override this behavior and re-use existing services, we set <code>useExisting: true</code>. This is helpful in cases where services  may take time to provision and initialize, where there is no risk in re-using them between pipeline runs, or where it is desirable to retain the service instance from the last pipeline run (for example, a database migration).
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="1-10-push-changes-to-github"><a class="link" href="#1-10-push-changes-to-github">1.10 Push changes to GitHub</a></h5>
<div class="paragraph">
<p>Push the preceding changes to GitHub. You should be pushing the following to each of the two application repositories:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>A new app manifest file</p>
</li>
<li>
<p>A new cloud-pipelines manifest file</p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="1-11-create-cloud-foundry-orgs-and-spaces"><a class="link" href="#1-11-create-cloud-foundry-orgs-and-spaces">1.11 Create Cloud Foundry Orgs and Spaces</a></h5>
<div class="paragraph">
<p>Cloud Pipelines requires that the Cloud Foundry test, stage, and prod spaces exist before a pipeline is run. If you wish, you can use different foundations, orgs, and users for each. For simplicity, in this example, we use a single foundation (PWS), a single org, and a single user.</p>
</div>
<div class="paragraph">
<p>You can name the orgs and spaces anything you like. Each app requires its own test space. The stage and prod spaces are shared.</p>
</div>
<div class="paragraph">
<p>For this example, use the following commands to create spaces:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">cf create-space scp-test-greeting-ui
cf create-space scp-test-fortune-service
cf create-space scp-stage
cf create-space scp-prod</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="1-12-create-cloud-foundry-stage-and-prod-service-instances"><a class="link" href="#1-12-create-cloud-foundry-stage-and-prod-service-instances">1.12 Create Cloud Foundry Stage and Prod Service Instances</a></h5>
<div class="paragraph">
<p>Cloud Pipelines dynamically creates the services in the test spaces, as defined by the <code>cloud-pipelines.yml</code> file we created previously. Optionally, you can add a second section to the <code>cloud-pipelines.yml</code> file for the stage environment, and these are created dynamically as well. However, you must always crate prod manually.</p>
</div>
<div class="paragraph">
<p>For this example, we create both the stage and prod services manually.</p>
</div>
<div class="paragraph">
<p>Create the services listed in the application manifest files in both <code>scp-stage</code> and <code>scp-prod</code>.</p>
</div>
</div>
<div class="sect4">
<h5 id="1-13-update-the-cloud-pipelines-credentials-file"><a class="link" href="#1-13-update-the-cloud-pipelines-credentials-file">1.13 Update the Cloud Pipelines Credentials File</a></h5>
<div class="paragraph">
<p>Update the <code>greeting-ui</code> and <code>fortune-service</code> credentials files with Cloud Foundry information. Replace values in the next example as appropriate for your Cloud Foundry environment.</p>
</div>
<div class="paragraph">
<p>Notice that the test space name specified is a prefix, unlike the stage and prod space names, which are literals. Cloud Pipelines append the application name to the test space name, thereby matching the test space names we created manually. The stage and prod space names are not prefixes and are not altered by Cloud Pipelines.</p>
</div>
<div class="paragraph">
<p>Note also the <code>paas-hostname-uuid</code>. The value is included in each created route. This value is optional, but it is useful in shared or multi-tenant environments (such as PWS), as it helps to ensure routes are unique. Change it to a unique uuid.</p>
</div>
<div class="paragraph">
<p>The following example shows an updated credentials file:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">pipeline-descriptor: cloud-pipelines.yml

paas-type: cf

paas-hostname-uuid: cyi

# test values
paas-test-api-url: https://api.run.pivotal.io
paas-test-username: ciberkleid@pivotal.io
paas-test-password: secret
paas-test-org: S1Pdemo12
paas-test-space-prefix: scp-test

# stage values
paas-stage-api-url: https://api.run.pivotal.io
paas-stage-username: ciberkleid@pivotal.io
paas-stage-password: my-super-secret-password
paas-stage-org: S1Pdemo12
paas-stage-space: scp-stage

# prod values
paas-prod-api-url: https://api.run.pivotal.io
paas-prod-username: ciberkleid@pivotal.io
paas-prod-password: my-super-secret-password
paas-prod-org: S1Pdemo12
paas-prod-space: scp-prod</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="1-14-update-the-concourse-pipeline-with-updated-credentials-files"><a class="link" href="#1-14-update-the-concourse-pipeline-with-updated-credentials-files">1.14 Update the Concourse Pipeline with Updated Credentials Files</a></h5>
<div class="paragraph">
<p>Set the Concourse pipelines again, as we did previously, to update them with the values added to the credentials files. The test, stage, and prod jobs should all now succeed and result in output similar to the following image:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/concourse_test_stage_prod_success.png" alt="concourse test stage prod success">
</div>
<div class="title">Figure 7. Test, Stage, &amp; Prod Success</div>
</div>
<div class="paragraph">
<p>On Cloud Foundry, you should now see the apps deployed in the test, stage, and prod spaces. The following image shows the deployment of <code>fortune-service</code> to its dedicated test space:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/cf_test_and_prod_deployed.png" alt="cf test and prod deployed">
</div>
<div class="title">Figure 8. Cloud Foundry Test and Prod Deployment</div>
</div>
<div class="paragraph">
<p>Notice that the five services declared in its manifest files (<code>cloud-pipelines.yml</code> for provisioning and <code>manifest.yml</code> for binding) have also been automatically provisioned. The image also shows the deployment of the same app to the shared prod space. Notice that the instance of the previous version has been renamed as <code>venerable</code> and stopped. If a rollback were deemed necessary, the <code>prod-rollback</code> job in the pipeline could be triggered to remove the currently running version, remove the <code>prod/&lt;version_number&gt;</code> tag from GitHub, and re-start the former (<code>venerable</code>) version.</p>
</div>
</div>
<div class="sect4">
<h5 id="stage-one-recap-and-next-steps"><a class="link" href="#stage-one-recap-and-next-steps">Stage One Recap and Next Steps</a></h5>
<div class="paragraph">
<p>What have we accomplished?</p>
</div>
<div class="ulist">
<ul>
<li>
<p>By adding the basic scaffolding needed to enable Cloud Pipelines to manage the lifecycle of <code>greeting-ui</code> and <code>fortune-service</code> from source code commit to production deploy, we have made it possible for the application development teams to instantly and easily create pipelines for each application by using a common, standardized template.</p>
</li>
<li>
<p>We can count on the pipelines to:</p>
<div class="ulist">
<ul>
<li>
<p>Automatically provision services in test spaces and, optionally, in stage spaces as well.</p>
</li>
<li>
<p>Dynamically clean up the test spaces between pipeline executions.</p>
</li>
<li>
<p>Upload the app artifacts to the maven repo (for example, Bintray).</p>
</li>
<li>
<p>Tag the git repositories with <code>dev/&lt;app_name&gt;/&lt;version_number&gt;</code> and <code>prod/&lt;app_name&gt;/&lt;version_number&gt;</code>.</p>
</li>
</ul>
</div>
</li>
<li>
<p>After each successful pipeline run, we can, if necessary, to roll back to the last deployed version byusing the <code>prod-rollback</code> job.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>These accomplishments are extremely valuable. However, to derive confidence and reliability from the pipelines, we need to incorporate testing. We do this in Stage Two of the application migration.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="tutorial-stage-two"><a class="link" href="#tutorial-stage-two">Stage Two: Tests</a></h4>
<div class="paragraph">
<p>In this stage, we enable Cloud Pipelines to execute tests so that we can increase confidence in the code being deployed. We do so by adding test profiles to the <code>pom.xml</code> files and then organizing or adding tests in a way that corresponds to the profiles. By doing so, we establish standards around testing across development teams in the enterprise.</p>
</div>
<div class="paragraph">
<p>We will also enable database schema versioning in this stage, thereby providing the foundation for rollback testing during schema changes.</p>
</div>
<div class="sect4">
<h5 id="2-1-add-maven-profiles"><a class="link" href="#2-1-add-maven-profiles">2.1 Add Maven Profiles</a></h5>
<div class="paragraph">
<p>For both <code>greeting-ui</code> and <code>fortune-service</code>, add a <code>profiles</code> section to the <code>pom.xml</code> file, as shown in the next listing. Note that we are adding four profiles:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>default</code></p>
<div class="ulist">
<ul>
<li>
<p>For unit and integration tests. Note that this profile includes all tests except those that are explicitly called by the smoke and e2e profiles.</p>
</li>
<li>
<p>Tests matching this profile run during the build-and-upload job.</p>
</li>
</ul>
</div>
</li>
<li>
<p><code>apicompatibility</code></p>
<div class="ulist">
<ul>
<li>
<p>For ensuring backward compatibility in case of API changes. Note that this is not effective until Stage Three, when we will add contracts. However, we add this profile now to ensure the api compatibility check during the <code>build-and-upload</code> job does not run other tests.</p>
</li>
</ul>
</div>
</li>
<li>
<p><code>smoke</code></p>
<div class="ulist">
<ul>
<li>
<p>For tests to be run against the application deployed in the test space.</p>
</li>
</ul>
</div>
</li>
<li>
<p><code>e2e</code></p>
<div class="ulist">
<ul>
<li>
<p>For tests to be run against the application deployed in the stage space.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>The following listing shows the necessary <code>profiles</code> element of a <code>pom.xml</code> file:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">  &lt;profiles&gt;
    &lt;profile&gt;
      &lt;id&gt;default&lt;/id&gt;
      &lt;activation&gt;
        &lt;activeByDefault&gt;true&lt;/activeByDefault&gt;
      &lt;/activation&gt;
      &lt;build&gt;
        &lt;plugins&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
            &lt;artifactId&gt;maven-surefire-plugin&lt;/artifactId&gt;
            &lt;configuration&gt;
              &lt;includes&gt;
                &lt;include&gt;**/*Tests.java&lt;/include&gt;
                &lt;include&gt;**/*Test.java&lt;/include&gt;
              &lt;/includes&gt;
              &lt;excludes&gt;
                &lt;exclude&gt;**/smoke/**&lt;/exclude&gt;
                &lt;exclude&gt;**/e2e/**&lt;/exclude&gt;
              &lt;/excludes&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;
            &lt;artifactId&gt;spring-boot-maven-plugin&lt;/artifactId&gt;
          &lt;/plugin&gt;
        &lt;/plugins&gt;
      &lt;/build&gt;
    &lt;/profile&gt;
    &lt;profile&gt;
      &lt;id&gt;apicompatibility&lt;/id&gt;
      &lt;build&gt;
        &lt;plugins&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
            &lt;artifactId&gt;maven-surefire-plugin&lt;/artifactId&gt;
            &lt;configuration&gt;
              &lt;includes&gt;
                &lt;include&gt;**/contracttests/**/*Tests.java&lt;/include&gt;
                &lt;include&gt;**/contracttests/**/*Test.java&lt;/include&gt;
              &lt;/includes&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
        &lt;/plugins&gt;
      &lt;/build&gt;
    &lt;/profile&gt;
    &lt;profile&gt;
      &lt;id&gt;smoke&lt;/id&gt;
      &lt;build&gt;
        &lt;plugins&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
            &lt;artifactId&gt;maven-surefire-plugin&lt;/artifactId&gt;
            &lt;configuration&gt;
              &lt;includes&gt;
                &lt;include&gt;smoke/**/*Tests.java&lt;/include&gt;
                &lt;include&gt;smoke/**/*Test.java&lt;/include&gt;
              &lt;/includes&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
        &lt;/plugins&gt;
      &lt;/build&gt;
    &lt;/profile&gt;
    &lt;profile&gt;
      &lt;id&gt;e2e&lt;/id&gt;
      &lt;build&gt;
        &lt;plugins&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
            &lt;artifactId&gt;maven-surefire-plugin&lt;/artifactId&gt;
            &lt;configuration&gt;
              &lt;includes&gt;
                &lt;include&gt;e2e/**/*Tests.java&lt;/include&gt;
                &lt;include&gt;e2e/**/*Test.java&lt;/include&gt;
              &lt;/includes&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
        &lt;/plugins&gt;
      &lt;/build&gt;
    &lt;/profile&gt;
  &lt;/profiles&gt;</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="2-2-add-and-organize-tests"><a class="link" href="#2-2-add-and-organize-tests">2.2 Add and Organize Tests</a></h5>
<div class="paragraph">
<p>Next, we ensure that we have a matching test package structure in our apps, as the following image shows:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/test_package_structure.png" alt="test package structure">
</div>
<div class="title">Figure 9. Test Package Structure</div>
</div>
<div class="paragraph">
<p>Note that we are creating matching packages only for the default, smoke, and e2e profiles. We will address the package for the <code>apicompatibility</code> profile in Stage Three.</p>
</div>
<div class="paragraph">
<p>When working with your own applications, if you have existing tests, you would move the files into one of these packages now and rename them so that they are included by the filters declared in the profiles (that is, the file names end in <code>Test.java</code> or <code>Tests.java</code>)</p>
</div>
<div class="paragraph">
<p>In the case of our sample apps, we have no tests, so we add some now.</p>
</div>
<div class="sect5">
<h6 id="fortune-service-default-tests"><a class="link" href="#fortune-service-default-tests"><code>fortune-service</code> Default Tests</a></h6>
<div class="paragraph">
<p>Add your unit and integration tests so that they match the default profile, as defined in the <code>fortune-service</code> <code>pom.xml</code> file. These are run on Concourse against the <code>fortune-service</code> application that runs on the Concourse worker in the <code>build-and-upload</code> job.</p>
</div>
<div class="paragraph">
<p>As an example, we add two tests, one that loads the context and another that verifies the number of rows expected in the database. The following example defines these tests:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package io.pivotal;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

import org.springframework.jdbc.core.JdbcTemplate;
import static org.assertj.core.api.Assertions.assertThat;

import static org.junit.Assert.*;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = FortuneServiceApplication.class)
public class FortuneServiceApplicationTests {

    @Test
    public void contextLoads() throws Exception {

    }

    @Autowired
    private JdbcTemplate template;

    @Test
    public void testDefaultSettings() throws Exception {
        assertThat(this.template.queryForObject("SELECT COUNT(*) from FORTUNE",
                Integer.class)).isEqualTo(7);
    }

}</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="fortune-service-smoke-tests"><a class="link" href="#fortune-service-smoke-tests"><code>fortune-service</code> Smoke Tests</a></h6>
<div class="paragraph">
<p>Add your smoke tests so that they match the smoke profile, as defined in the <code>fortune-service</code> <code>pom.xml</code> file. These run on Concourse against the <code>fortune-service</code> application deployed in the Cloud Foundry <code>scp-test-fortune-service</code> space. Two versions of these tests are executed against the application:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>the current version, in the <code>test-smoke</code> job.</p>
</li>
<li>
<p>the latest prod version, in the <code>test-rollback-smoke</code> job.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The following image shows the tests in Concourse:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/fortune_service_smoke_tests.png" alt="fortune service smoke tests">
</div>
<div class="title">Figure 10. fortune-service Smoke Tests</div>
</div>
<div class="paragraph">
<p>In the test environment, we choose to verify that <code>fortune-service</code> is retrieving a fortune from <code>fortune-db</code>, and not returning its Hystrix fallback response. The following example defines this test:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package smoke;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SmokeTests.class,
        webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class SmokeTests {

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity&lt;String&gt; response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback response
		BDDAssertions.then(response.getBody()).doesNotContain("The fortuneteller will be back soon.");
	}

}</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="fortune-service-end-to-end-e2e-tests"><a class="link" href="#fortune-service-end-to-end-e2e-tests"><code>fortune-service</code> End-to-end (<code>e2e</code>) Tests</a></h6>
<div class="paragraph">
<p>Add your end-to-end tests so that they match the <code>e2e</code> profile, as defined in the <code>fortune-service</code> <code>pom.xml</code> file. These tests run on Concourse against the <code>fortune-service</code> application deployed in the Cloud Foundry <code>scp-stage</code> space. This space is shared, so we assume <code>greeting-ui</code> is also present.</p>
</div>
<div class="paragraph">
<p>The following image shows the tests in Concourse:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/fortune_service_e2e_tests.png" alt="fortune service e2e tests">
</div>
<div class="title">Figure 11. fortune-service E2E Tests</div>
</div>
<div class="paragraph">
<p>In the <code>e2e</code> environment, we choose to use a string replacement to obtain the URL for <code>greeting-ui</code>. We also choose to verify that we hit <code>fortune-db</code> and do not receive Hystrix fallback responses from either application. The following example shows this test:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package e2e;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = E2eTests.class,
		webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class E2eTests {

	// The app is running in CF but the tests are executed from Concourse worker,
	// so the test will deduce the url to greeting-ui: it will assume the same host
	// as fortune-service, and simply replace "fortune-service" with "greeting-ui" in the url

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity&lt;String&gt; response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl.replace("fortune-service", "greeting-ui") + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback responses from both fortune and greeting
		BDDAssertions.then(response.getBody()).doesNotContain("This fortune is no good. Try another.").doesNotContain("The fortuneteller will be back soon.");
	}

}</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="greeting-ui-default-tests"><a class="link" href="#greeting-ui-default-tests"><code>greeting-ui</code> Default Tests</a></h6>
<div class="paragraph">
<p>Add your unit and integration tests so that they match the default profile, as defined in the <code>greeting-ui</code> <code>pom.xml</code> file. These run on Concourse against the <code>greeting-ui</code> application that runs on the Concourse worker in the <code>build-and-upload</code> job.</p>
</div>
<div class="paragraph">
<p>As an example, we add one test that loads the context:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package io.pivotal;

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = GreetingUIApplication.class)
public class GreetingUIApplicationTests {

    @Test
    public void contextLoads() throws Exception {

    }

}</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="greeting-ui-smoke-tests"><a class="link" href="#greeting-ui-smoke-tests"><code>greeting-ui</code> Smoke Tests</a></h6>
<div class="paragraph">
<p>Add your smoke tests so that they match the smoke profile, as defined in the <code>greeting-ui</code> <code>pom.xml</code> file. These run on Concourse against the <code>greeting-ui</code> application deployed in the Cloud Foundry <code>scp-test-greeting-ui</code> space. Two versions of these tests run against the app:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>The current version, in the <code>test-smoke</code> job.</p>
</li>
<li>
<p>The latest prod version, in the <code>test-rollback-smoke</code> job.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The following image shows the tests in Concourse:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/greeting_ui_smoke_tests.png" alt="greeting ui smoke tests">
</div>
<div class="title">Figure 12. greeting-ui Smoke Tests</div>
</div>
<div class="paragraph">
<p>Since <code>fortune-service</code> is not deployed to the <code>scp-test-greeting-ui</code> space, we expect to receive the Hystrix fallback response defined in <code>greeting-ui</code>. Hence, our smoke test validates that condition:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package smoke;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SmokeTests.class,
        webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class SmokeTests {

    @Value("${application.url}") String applicationUrl;

    RestTemplate restTemplate = new RestTemplate();

    @Test
    public void should_return_a_fallback_fortune() {
        ResponseEntity&lt;String&gt; response = this.restTemplate
                .getForEntity("http://" + this.applicationUrl + "/", String.class);

        BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

        // Expect the hystrix fallback response
        BDDAssertions.then(response.getBody()).contains("This fortune is no good. Try another.");
    }

}</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="greeting-ui-end-to-end-e2e-tests"><a class="link" href="#greeting-ui-end-to-end-e2e-tests"><code>greeting-ui</code> End-to-end (<code>e2e</code>) Tests</a></h6>
<div class="paragraph">
<p>Add your end-to-end tests so that they match the <code>e2e</code> profile, as defined in the <code>greeting-ui</code> <code>pom.xml</code> file. These run on Concourse against the <code>greeting-ui</code> application deployed in the Cloud Foundry <code>scp-stage</code> space. This space is shared, so we assume <code>fortune-service</code> is also present.</p>
</div>
<div class="paragraph">
<p>The following image shows the tests in Concourse:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/greeting_ui_e2e_tests.png" alt="greeting ui e2e tests">
</div>
<div class="title">Figure 13. greeting-ui E2E Tests</div>
</div>
<div class="paragraph">
<p>In the <code>e2e</code> environment, we choose to verify that we hit <code>fortune-service</code> and do not receive the Hystrix fallback response from <code>greeting-ui</code>. The following example shows the test:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package e2e;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = E2eTests.class,
		webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class E2eTests {

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity&lt;String&gt; response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback response
		BDDAssertions.then(response.getBody()).doesNotContain("This fortune is no good. Try another.");
	}

}</code></pre>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="2-3-enable-database-versioning"><a class="link" href="#2-3-enable-database-versioning">2.3 Enable Database Versioning</a></h5>
<div class="paragraph">
<p>At this point, we also incorporate <a href="https://flywaydb.org/">Flyway</a>, an OSS database migration tool, to track database schema versions and handle schema changes and data loading.</p>
</div>
<div class="paragraph">
<p>This change needs to be made only to <code>fortune-service</code>, since <code>fortune-service</code> owns the interaction with <code>fortune-db</code>.</p>
</div>
<div class="sect5">
<h6 id="add-flyway-dependency"><a class="link" href="#add-flyway-dependency">Add Flyway Dependency</a></h6>
<div class="paragraph">
<p>We first add the Flyway dependency to the <code>fortune-service</code> <code>pom.xml</code>. We need not add a version as Spring Boot takes care of that for us. The following listing shows the Flyway dependency:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">    &lt;dependency&gt;
      &lt;groupId&gt;org.flywaydb&lt;/groupId&gt;
      &lt;artifactId&gt;flyway-core&lt;/artifactId&gt;
    &lt;/dependency&gt;
    &lt;dependency&gt;</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="create-flyway-migration"><a class="link" href="#create-flyway-migration">Create Flyway Migration</a></h6>
<div class="paragraph">
<p>Next, we create a migration directory and our initial migration file, following Flyway&#8217;s file naming convention. The following image shows the name of the file in context:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/fortune_service_flyway_file_name.png" alt="fortune service flyway file name">
</div>
<div class="title">Figure 14. fortune-service Flyway File Name</div>
</div>
<div class="paragraph">
<p>Note that the filename specifies the version (<code>V1</code>), followed by two underscore characters.</p>
</div>
<div class="paragraph">
<p>We place our <code>CREATE TABLE</code> and <code>INSERT</code> statements in our <code>src/main/resources/db/migration/V1__init.sql</code> file, as the following listing shows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-sql hljs" data-lang="sql">CREATE TABLE fortune (
  id BIGINT PRIMARY KEY AUTO_INCREMENT,
  text varchar(255) not null
);

INSERT INTO fortune (text) VALUES ('Do what works.');

INSERT INTO fortune (text) VALUES ('Do the right thing.');

INSERT INTO fortune (text) VALUES ('Always be kind.');

INSERT INTO fortune (text) VALUES ('You learn from your mistakes... You will learn a lot today.');

INSERT INTO fortune (text) VALUES ('You can always find happiness at work on Friday.');

INSERT INTO fortune (text) VALUES ('You will be hungry again in one hour.');

INSERT INTO fortune (text) VALUES ('Today will be an awesome day!');</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="disable-jpa-ddl-initialization"><a class="link" href="#disable-jpa-ddl-initialization">Disable JPA DDL Initialization</a></h6>
<div class="paragraph">
<p>Because we rely on Flyway to create and populate the schema, we need to disable JPA-based database initialization. We can set <code>ddl-auto</code> to <code>validate</code>, which validates the schema against the application entities and throws an error in case of a mismatch but does not actually generate the schema. The following snippet shows how to do so:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">spring:
  jpa:
    hibernate:
      ddl-auto: validate</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>There are a few options for where to store the <code>ddl-auto</code> configuration, both in terms of location (in the <code>fortune-service</code> app or on the <code>app-config</code> GitHub repo) and in terms of file name. For this example, update the <code>application.yml</code> in the <code>fortune-service</code> app for local testing. Additionally, save these values in a new file called <code>application-flyway.yml</code> on your fork of <a href="https://github.com/ciberkleid/app-config"><code>app-config</code></a>.</p>
</div>
<div class="paragraph">
<p>By convention, <code>fortune-service</code> picks up the configurations in <code>application-flyway.yml</code> if the string <code>flyway</code> is in the list of active Spring profiles. Consequently, we can add <code>flyway</code> to the <code>SPRING_PROFILES_ACTIVE</code> environment variable in the <code>fortune-service</code> <code>manifest.yml</code>, as the following listing shows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">---
applications:
- name: fortune-service
  timeout: 120
  services:
  - fortune-db
  - config-server
  - cloud-bus
  - service-registry
  - circuit-breaker-dashboard
  env:
    SPRING_PROFILES_ACTIVE: flyway
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="remove-non-flyway-data-loading"><a class="link" href="#remove-non-flyway-data-loading">Remove Non-Flyway Data Loading</a></h6>
<div class="paragraph">
<p>We can now remove the old code that populated the database. In our sample app, this was found in the <code>io.pivotal.FortuneServiceApplication</code> class. The following listing shows the code we now remove:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">@Bean
    CommandLineRunner loadDatabase(FortuneRepository fortuneRepo) {
        return args -&gt; {
//            logger.debug("loading database..");
//            fortuneRepo.save(new Fortune(1L, "Do what works."));
//            fortuneRepo.save(new Fortune(2L, "Do the right thing."));
//            fortuneRepo.save(new Fortune(3L, "Always be kind."));
//            fortuneRepo.save(new Fortune(4L, "You learn from your mistakes... You will learn a lot today."));
//            fortuneRepo.save(new Fortune(5L, "You can always find happiness at work on Friday."));
//            fortuneRepo.save(new Fortune(6L, "You will be hungry again in one hour."));
//            fortuneRepo.save(new Fortune(7L, "Today will be an awesome day!"));
            logger.debug("record count: {}", fortuneRepo.count());
            fortuneRepo.findAll().forEach(x -&gt; logger.debug(x.toString()));
        };

    }</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>We also no longer need the <code>Fortune</code> entity constructors, so we can comment these out in the <code>io.pivotal.fortune.Fortune</code> class as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">//    public Fortune() {
//    }
//
//    public Fortune(Long id, String text) {
//        super();
//        this.id = id;
//        this.text = text;
//    }</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="flyway-integration-summary"><a class="link" href="#flyway-integration-summary">Flyway Integration Summary</a></h6>
<div class="paragraph">
<p>With that, we have completed the setup for Flyway, and our database schema is now versioned. From this point onward, Spring Boot calls <code>Flyway.migrate()</code> to perform the database migration. As long as we follow Flyway conventions for future schema changes, Flyway takes care of tracking the schema version and migrating the database for us.</p>
</div>
<div class="paragraph">
<p>From a rollback perspective, Cloud Pipelines includes two jobs in the <code>test</code> phase (<code>test-rollback-deploy</code> and <code>test-rollback-smoke</code>), wherein it validates that the latest prod jar works against the newly updated database. The purpose is to ensure that we can roll back the application in prod if a problem is discovered after the prod database schema has been updated and avoid the burden of rolling back the database.</p>
</div>
<div class="paragraph">
<p>See <a href="https://docs.spring.io/spring-boot/docs/current/reference/html/howto-database-initialization.html#howto-use-a-higher-level-database-migration-tool">Spring Boot database initialization with Flyway</a> for further information, including Flyway configuration options.</p>
</div>
</div>
</div>
<div class="sect4">
<h5 id="2-4-push-changes-to-github"><a class="link" href="#2-4-push-changes-to-github">2.4 Push Changes to GitHub</a></h5>
<div class="paragraph">
<p>For <code>greeting-ui</code>, you should push the following new or modified files:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>pom.xml</code></p>
</li>
<li>
<p><code>src/test/java/e2e/E2eTests.java</code></p>
</li>
<li>
<p><code>src/test/java/io/pivotal/GreetingUIApplicationTests.java</code></p>
</li>
<li>
<p><code>src/test/java/smoke/SmokeTests.java</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For <code>fortune-service</code>, you should be pushing the following new or modified files:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>pom.xml</code></p>
</li>
<li>
<p><code>src/test/java/e2e/E2eTests.java</code></p>
</li>
<li>
<p><code>src/test/java/io/pivotal/FortuneServiceApplicationTests.java</code></p>
</li>
<li>
<p><code>src/test/java/smoke/SmokeTests.java</code></p>
</li>
<li>
<p><code>src/main/resources/db/migration/V1__init.sql</code></p>
</li>
<li>
<p><code>src/main/resources/application.yml</code></p>
</li>
<li>
<p><code>manifest.yml</code></p>
</li>
<li>
<p><code>src/main/java/io/pivotal/FortuneServiceApplication.java</code></p>
</li>
<li>
<p><code>src/main/java/io/pivotal/fortune/Fortune.java</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>For <code>app-config</code>, you should be pushing the following new:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>application-flyway.yml</code></p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="2-5-re-run-the-pipelines"><a class="link" href="#2-5-re-run-the-pipelines">2.5 Re-run the Pipelines</a></h5>
<div class="paragraph">
<p>Run through the pipelines again and view the output for the jobs that run the default, smoke, and end-to-end (<code>e2e</code>) tests. You should see that the tests we added in this stage were run.</p>
</div>
<div class="paragraph">
<p>As you run through the pipelines a second time, you should see the smoke tests from the latest prod version run against the database in the <code>test-rollback-smoke</code> job. In this case, there is no schema upgrade. Nonetheless, the tests confirm that the latest prod version of the app can be used with the current database schema.</p>
</div>
<div class="paragraph">
<p>You can see the database version information stored in the database by Flyway either by querying the database itself or by hitting the flyway endpoint on the <code>fortune-service</code> URL. The following image shows an example from the <code>scp-stage</code> environment:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/fortune_service_flyway_schema_info.png" alt="fortune service flyway schema info">
</div>
<div class="title">Figure 15. fortune-service Flyway Schema Info</div>
</div>
</div>
<div class="sect4">
<h5 id="stage-two-recap-and-next-steps"><a class="link" href="#stage-two-recap-and-next-steps">Stage Two Recap and Next Steps</a></h5>
<div class="paragraph">
<p>What have we accomplished?</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Increased the effectiveness of the pipelines and our confidence in them by integrating our applications with the testing strategy built into Cloud Pipelines.</p>
</li>
<li>
<p>Established a standard approach to organizing tests, which brings consistency within and across development teams.</p>
</li>
<li>
<p>Enabled auto-managed database versioning and backward compatibility testing that alleviates database schema management throughout the release management lifecycle.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>We can now add any unit, integration, smoke, and end-to-end tests to our code base and have a high level of reliability and confidence in our pipelines. We are also better positioned to ensure that our development teams conform to these practices, given the structure established by Cloud Pipelines and the fast feedback and visibility we gain from the pipelines as they execute the tests.</p>
</div>
<div class="paragraph">
<p>However, we could benefit further by incorporating contracts to define and test the API integration points between applications. We do this in Stage Three of the application migration.</p>
</div>
</div>
</div>
<div class="sect3">
<h4 id="tutorial-stage-three"><a class="link" href="#tutorial-stage-three">Stage Three: Contracts</a></h4>
<div class="paragraph">
<p>In this stage, we introduce contract-based programming practices into our sample application. Doing so improves API management capabilities, including defining, communicating, and testing API semantics. It also lets us catch breaking API changes (that is, we can validate API backward compatibility) in the build phase. This extends the effectiveness of the pipelines, encourages better communication and programming practices across development teams, and provides faster feedback to developers.</p>
</div>
<div class="paragraph">
<p>We will integrate Spring Cloud Contract and add contracts, stubs, and a stub runner. We will also now complete and make use of the <code>apicompatibility</code> profile defined in <a href="#tutorial-stage-two">Stage Two: Tests</a>.</p>
</div>
<div class="sect4">
<h5 id="3-1-create-a-contract"><a class="link" href="#3-1-create-a-contract">3.1 Create a Contract</a></h5>
<div class="paragraph">
<p>We start by creating the contract for the interaction between <code>greeting-ui</code> and <code>fortune-service</code>. The contract should describe the following expectation: <code>greeting-ui</code> makes a <code>GET</code> request to the root URL of <code>fortune-service</code> and expects a response with status 200 and a string (<code>new fortune</code>) in the body</p>
</div>
<div class="paragraph">
<p>We code this buy using groovy syntax as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-groovy hljs" data-lang="groovy">import org.springframework.cloud.contract.spec.Contract

Contract.make {
    description("""
should return a fortune string
""")
    request {
        method GET()
        url "/"
    }
    response {
        status 200
        body "new fortune"
    }
}</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Save this contract in the <code>fortune-service</code> code base in the following location (which is compliant with Spring Cloud Contract convention): <code>src/test/resources/contracts/&lt;service-name&gt;/&lt;contract-file&gt;</code>. The following image shows the contract file in its location within an IDE:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/fortune_service_contract_file.png" alt="fortune service contract file">
</div>
<div class="title">Figure 16. fortune-service Flyway Contract File</div>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
<div class="paragraph">
<p>You can optionally enable your IDE to assist with contract syntax by adding the Spring Cloud Contract Verifier to your <code>pom.xml</code> file. It is pluggable, and includes groovy and pact by default. The following element shows the dependency to add:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">    &lt;dependency&gt;
      &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
      &lt;artifactId&gt;spring-cloud-starter-contract-verifier&lt;/artifactId&gt;
      &lt;scope&gt;test&lt;/scope&gt;
    &lt;/dependency&gt;</code></pre>
</div>
</div>
</div>
</div>
</td>
</tr>
</table>
</div>
</div>
<div class="sect4">
<h5 id="3-2-create-a-base-class-for-contract-tests"><a class="link" href="#3-2-create-a-base-class-for-contract-tests">3.2 Create a Base Class for Contract Tests</a></h5>
<div class="paragraph">
<p>Now that we have a coded contract, we want to enable auto-generation of contract-based tests. The auto-generation, which we will configure in the next steps, requires a base class that stubs out the service that satisfies the API call, so that we can run the test without external dependencies (for example, the database). The objective is to focus on testing API semantics.</p>
</div>
<div class="paragraph">
<p>We create the base class in the <code>fortune-service</code> test package as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package io.pivotal.fortune;

import io.restassured.module.mockmvc.RestAssuredMockMvc;
import org.junit.Before;
import org.mockito.BDDMockito;

public class BaseClass {

    @Before
    public void setup() {
        FortuneService service = BDDMockito.mock(FortuneService.class);
        BDDMockito.given(service.getFortune()).willReturn("foo fortune");
        RestAssuredMockMvc.standaloneSetup(new FortuneController(service));
    }
}</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect4">
<h5 id="3-3-enable-automated-contract-based-testing"><a class="link" href="#3-3-enable-automated-contract-based-testing">3.3 Enable Automated Contract-based Testing</a></h5>
<div class="paragraph">
<p>Now that we have a contract and a base class, we can use the Spring Cloud Contract Maven plugin to auto-generate contract tests, stubs, and a stub jar.</p>
</div>
<div class="paragraph">
<p>First we add the Spring Cloud Contract version to the list of properties in the <code>fortune-service</code> <code>pom.xml</code> file, since we will reference it when we enable the Spring Cloud Contract maven plugin. To do so, we add the <code>&lt;spring-cloud-contract.version&gt;</code> to the <code>properties</code> element, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">  &lt;properties&gt;
...
    &lt;spring-cloud-contract.version&gt;1.2.1.RELEASE&lt;/spring-cloud-contract.version&gt;
...
&lt;/properties&gt;</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Next, we edit the <code>default</code> profile in the <code>fortune-service</code> <code>pom.xml</code> file to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Add a plugin block for Spring Cloud Contract maven plugin.</p>
</li>
<li>
<p>Configure it to use our base class (<code>io.pivotal.fortune.BaseClass</code>) to generate tests.</p>
</li>
<li>
<p>Configure it to place auto-generated tests in the <code>io.pivotal.fortune.contracttests</code> test package.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Note that the package of the contract tests is included by the <code>include</code> filter in the <code>default</code> profile, so these tests run against the application during the <code>build-and-upload</code> job. For <code>fortune-service</code>, this serves to validate that the application conforms to the contract.</p>
</div>
<div class="paragraph">
<p>The following listing shows the complete profile:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">    &lt;profile&gt;
      &lt;id&gt;default&lt;/id&gt;
      &lt;activation&gt;
        &lt;activeByDefault&gt;true&lt;/activeByDefault&gt;
      &lt;/activation&gt;
      &lt;build&gt;
        &lt;plugins&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
            &lt;artifactId&gt;maven-surefire-plugin&lt;/artifactId&gt;
            &lt;configuration&gt;
              &lt;includes&gt;
                &lt;include&gt;**/*Tests.java&lt;/include&gt;
                &lt;include&gt;**/*Test.java&lt;/include&gt;
              &lt;/includes&gt;
              &lt;excludes&gt;
                &lt;exclude&gt;**/smoke/**&lt;/exclude&gt;
                &lt;exclude&gt;**/e2e/**&lt;/exclude&gt;
              &lt;/excludes&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.springframework.boot&lt;/groupId&gt;
            &lt;artifactId&gt;spring-boot-maven-plugin&lt;/artifactId&gt;
          &lt;/plugin&gt;
          &lt;!--Spring Cloud Contract maven plugin --&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
            &lt;artifactId&gt;spring-cloud-contract-maven-plugin&lt;/artifactId&gt;
            &lt;version&gt;${spring-cloud-contract.version}&lt;/version&gt;
            &lt;extensions&gt;true&lt;/extensions&gt;
            &lt;configuration&gt;
              &lt;baseClassForTests&gt;io.pivotal.fortune.BaseClass&lt;/baseClassForTests&gt;
              &lt;basePackageForTests&gt;io.pivotal.fortune.contracttests&lt;/basePackageForTests&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
        &lt;/plugins&gt;
      &lt;/build&gt;
    &lt;/profile&gt;</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>When the app is built, the Spring Cloud Contract maven plugin also now produces a stub and a stub jar that contains the contract and stub. This stub jar is uploaded to Bintray, along with the usual app jar. As we see shortly, this stub jar can be used by the <code>greeting-ui</code> development team while they wait for <code>fortune-service</code> to be completed. In other words, this gives the <code>greeting-ui</code> development team a producer to test against that is based on a mutually agreed-upon contract without the lead time of having to wait for <code>fortune-service</code> team to implement anything more than a base class and without having to manually stub out calls to <code>fortune-service</code> based on arbitrary or static responses.</p>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
Package the project locally (run <code>mvn package</code>) to observe the tests, stubs, and stub jar that the Spring Cloud Contract Maven plugin generates. See the following image for reference:
</td>
</tr>
</table>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/fortune_service_generated_tests.png" alt="fortune service generated tests">
</div>
<div class="title">Figure 17. Generated Tests and Stubs</div>
</div>
</div>
<div class="sect4">
<h5 id="3-4-enable-backward-compatibility-api-check"><a class="link" href="#3-4-enable-backward-compatibility-api-check">3.4 Enable backward compatibility API check</a></h5>
<div class="paragraph">
<p>To enable Cloud Pipelines to catch any breaking API changes during the the API compatibility check in the <code>build-and-upload</code> job, we add the Spring Cloud Contract Maven plugin to the <code>apicompatibility</code> profile as well.</p>
</div>
<div class="paragraph">
<p>In this case, we want the plugin to generate tests based on contracts outside of the project (the ones from the latest prod version), so we configure the plugin to download the latest prod stub jar, which contains the old contract. The plugin uses the old contract and the specified base class (which, in our example, is the same as the one in the previous step) to generate contract tests. These tests are run against the new code to validate that it is still compatible with consumers that comply with the prior contract. This ensures backward compatibility for the API.</p>
</div>
<div class="paragraph">
<p>In short, we edit the <code>apicompatibility</code> profile in the <code>fortune-service</code> <code>pom.xml</code> file to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Add a plugin block for Spring Cloud Contract Maven plugin.</p>
</li>
<li>
<p>Configure it to download the latest prod stub jar from Bintray to obtain the old contract.</p>
</li>
<li>
<p>Configure it to use our base class (<code>io.pivotal.fortune.BaseClass</code>) to generate tests (we use the same one as in the prior step).</p>
</li>
<li>
<p>Configure it to place auto-generated tests in the <code>io.pivotal.fortune.contracttests</code> test package.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Note that the package of the contract tests matches the <code>include</code> filter in the <code>apicompatibility</code> profile, so these tests run against the app during the the API compatibility check during the <code>build-and-upload</code> job. For <code>fortune-service</code>, this serves to validate that the app conforms to the old contract.</p>
</div>
<div class="paragraph">
<p>The following listing shows the complete profile:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">    &lt;profile&gt;
      &lt;id&gt;apicompatibility&lt;/id&gt;
      &lt;build&gt;
        &lt;plugins&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
            &lt;artifactId&gt;maven-surefire-plugin&lt;/artifactId&gt;
            &lt;configuration&gt;
              &lt;includes&gt;
                &lt;include&gt;**/contracttests/**/*Tests.java&lt;/include&gt;
                &lt;include&gt;**/contracttests/**/*Test.java&lt;/include&gt;
              &lt;/includes&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
          &lt;!--Spring Cloud Contract maven plugin --&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
            &lt;artifactId&gt;spring-cloud-contract-maven-plugin&lt;/artifactId&gt;
            &lt;version&gt;${spring-cloud-contract.version}&lt;/version&gt;
            &lt;extensions&gt;true&lt;/extensions&gt;
            &lt;configuration&gt;
              &lt;contractsRepositoryUrl&gt;${repo.with.binaries}&lt;/contractsRepositoryUrl&gt;
              &lt;contractDependency&gt;
                &lt;groupId&gt;${project.groupId}&lt;/groupId&gt;
                &lt;artifactId&gt;${project.artifactId}&lt;/artifactId&gt;
                &lt;classifier&gt;stubs&lt;/classifier&gt;
                &lt;version&gt;${latest.production.version}&lt;/version&gt;
              &lt;/contractDependency&gt;
              &lt;contractsPath&gt;/&lt;/contractsPath&gt;
              &lt;baseClassForTests&gt;io.pivotal.fortune.BaseClass&lt;/baseClassForTests&gt;
              &lt;basePackageForTests&gt;io.pivotal.fortune.contracttests&lt;/basePackageForTests&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
        &lt;/plugins&gt;
      &lt;/build&gt;
    &lt;/profile&gt;</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Cloud Pipelines dynamically injects the values for <code>${repo.with.binaries}</code> and <code>${latest.production.version}</code>. You can run this locally by providing these values manually as system properties in the Maven command.</p>
</div>
</div>
<div class="sect4">
<h5 id="3-5-push-changes-to-github"><a class="link" href="#3-5-push-changes-to-github">3.5 Push Changes to GitHub</a></h5>
<div class="paragraph">
<p>All changes in Stage Three thus far are in <code>fortune-service</code>. At this point, you should be pushing the following new or modified files:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>pom.xml</code></p>
</li>
<li>
<p><code>src/test/resources/contracts/greeting-ui/shouldReturnAFortune.groovy</code></p>
</li>
<li>
<p><code>src/test/java/io/pivotal/fortune/BaseClass.java</code></p>
</li>
</ul>
</div>
</div>
<div class="sect4">
<h5 id="3-6-re-run-the-fortune-service-pipeline"><a class="link" href="#3-6-re-run-the-fortune-service-pipeline">3.6 Re-run the <code>fortune-service</code> Pipeline</a></h5>
<div class="paragraph">
<p>Run through the <code>fortune-service</code> pipeline to generate stubs. The following output from the <code>build-and-upload</code> job shows the auto-generation of tests and stubs:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/fortune_service_build_and_upload_test_and_stub_generation.png" alt="fortune service build and upload test and stub generation">
</div>
<div class="title">Figure 18. fortune-service build-and-upload Test and Stub Generation</div>
</div>
<div class="paragraph">
<p>You should also see output in the <code>build-and-upload</code> job that shows the execution of these tests against the code.</p>
</div>
<div class="paragraph">
<p>Additionally, you should see the stub jar uploaded to Bintray along with the usual app jar.</p>
</div>
<div class="paragraph">
<p>Finally, as you run through the pipeline a second time, you should see that the contract tests from the latest prod version run against the new code in the output of the the API compatibility check during the <code>build-and-upload</code> job. In this case, there is no API change. Nonetheless, the tests confirm that the latest prod version of the API can be used with the current code base.</p>
</div>
</div>
<div class="sect4">
<h5 id="3-7-enable-stubs-for-integration-tests"><a class="link" href="#3-7-enable-stubs-for-integration-tests">3.7 Enable Stubs for Integration Tests</a></h5>
<div class="paragraph">
<p>Now we turn our attention to <code>greeting-ui</code>.</p>
</div>
<div class="paragraph">
<p>The following image compares the path of a request through <code>greeting-ui</code> in the build phase, both with and without stubs:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/greeting_ui_build_flow.png" alt="greeting ui build flow">
</div>
<div class="title">Figure 19. greeting-ui Build Flow</div>
</div>
<div class="paragraph">
<p>Without stubs, we expect the response to be the Hystrix fallback response that is hard-coded in <code>greeting-ui</code>. With stubs, however, we can expect the response that was declared in the contract. In this case, the stubs are loaded into the <code>greeting-ui</code> process. This leads us to our next task: Loading the stubs produced by <code>fortune-service</code>.</p>
</div>
<div class="sect5">
<h6 id="enable-the-in-process-stub-runner"><a class="link" href="#enable-the-in-process-stub-runner">Enable the in-process Stub Runner</a></h6>
<div class="paragraph">
<p>To load the stubs into the <code>greeting-ui</code> process, we must enable the Spring Cloud Contract Stub Runner dependency. This dependency start ans in-process stub runner that automatically configures Wiremock.</p>
</div>
<div class="paragraph">
<p>Add the following dependency to the <code>greeting-ui</code> <code>pom.xml</code> file:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">&lt;dependency&gt;
 &lt;groupId&gt;org.springframework.cloud&lt;/groupId&gt;
 &lt;artifactId&gt;spring-cloud-starter-contract-stub-runner&lt;/artifactId&gt;
 &lt;scope&gt;test&lt;/scope&gt;
&lt;/dependency&gt;</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="add-integration-tests-aligned-with-the-contract"><a class="link" href="#add-integration-tests-aligned-with-the-contract">Add Integration Tests Aligned with the Contract</a></h6>
<div class="paragraph">
<p>Next, we add integration tests to <code>greeting-ui</code> to test for the expected response declared in the contract.</p>
</div>
<div class="paragraph">
<p>Add the following class to the <code>test</code> package in <code>greeting-ui</code>:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package io.pivotal.fortune;

import io.pivotal.GreetingUIApplication;
import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.cloud.contract.stubrunner.spring.AutoConfigureStubRunner;
import org.springframework.test.context.junit4.SpringRunner;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = GreetingUIApplication.class, webEnvironment = SpringBootTest.WebEnvironment.NONE,
        properties = {"spring.application.name=greeting-ui", "spring.cloud.circuit.breaker.enabled=false", "hystrix.stream.queue.enabled=false"})
@AutoConfigureStubRunner(ids = {"io.pivotal:fortune-service:1.0.0.M1-20180102_203542-VERSION"},
        repositoryRoot = "${REPO_WITH_BINARIES}"
        //workOffline = true
)

public class FortuneServiceTests {

    @Autowired FortuneService fortuneService;

    @Test
    public void shouldSendRequestToFortune() {
        // when
        String fortune = fortuneService.getFortune();
        // then
        BDDAssertions.then(fortune).isEqualTo("foo fortune");
    }

}</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>At this point, we can get through the build phase for <code>greeting-ui</code>, and the integration tests run against the <code>fortune-service</code> stubs that runs in the <code>greeting-ui</code> process on the Concourse worker.</p>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
Notice the configuration of <code>@AutoConfigureStubRunner</code>. You can replace the version with a <code>+</code> sign if using Artifactory or Nexus and it automatically chooses the latest available version on the maven repo.
</td>
</tr>
</table>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
Setting <code>workOffline=true</code> (commented out but shown earlier for informational purposes) would make the stub runner get the stubs from the local Maven repository. This is useful for local testing. Alternatively, set the <code>$REPO_WITH_BINARIES</code> environment variable to the value used in the credentials file before doing a local Maven build. Then the local build will download the stubs from your remote Maven repository (for example, Bintray).
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect4">
<h5 id="3-8-enable-stubs-for-smoke-tests"><a class="link" href="#3-8-enable-stubs-for-smoke-tests">3.8 Enable Stubs for Smoke Tests</a></h5>
<div class="paragraph">
<p>The following image compares the path of a request through <code>greeting-ui</code> in the test phase, both with and without stubs. Note that in the build phase, where the app process runS on the Concourse worker, we ran the stubs in the same process. In the test environment (Cloud Foundry), we run the stubs in a separate process by using a standalone stub runner application. The following image shows the test flow for the <code>greeting-ui</code> application:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/greeting_ui_test_flow.png" alt="greeting ui test flow">
</div>
<div class="title">Figure 20. greeting-ui Test Flow</div>
</div>
<div class="paragraph">
<p>As in the build phase, without stubs, we expect the response to be the Hystrix fallback response that is hard-coded in <code>greeting-ui</code>. With stubs, however, we can expect the response that was declared in the contract.</p>
</div>
<div class="paragraph">
<p>We rely on Cloud Pipelines to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Deploy a stub runner application.</p>
</li>
<li>
<p>Provide the stub runner application with the necessary information to download the stubs.</p>
</li>
<li>
<p>Open a port on the stub runner application for each stub.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>We rely on the stub runner application to:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Download the stubs from our Maven repository (Bintray).</p>
</li>
<li>
<p>Expose each stub on a separate port.</p>
</li>
<li>
<p>Register each stub in the Service Discovery server.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>The following steps describe how to configure stubs for smoke tests.</p>
</div>
<div class="sect5">
<h6 id="provide-a-stand-alone-stub-runner-app-jar"><a class="link" href="#provide-a-stand-alone-stub-runner-app-jar">Provide a Stand-alone Stub Runner App Jar</a></h6>
<div class="paragraph">
<p>In the <a href="#tutorial-prep">Prep step</a> for this tutorial, you cloned the <a href="https://github.com/spring-cloud-samples/cloudfoundry-stub-runner-boot"><code>cloudfoundry-stub-runner-boot</code></a> repo to your local machine. The next step is to build this application and upload it to Bintray to make the jar available to Cloud Pipelines.</p>
</div>
<div class="paragraph">
<p>As mentioned in <a href="#tutorial-stage-one">Stage One: Scaffolding</a> of this tutorial, Bintray requires that a package exist before any application artifacts can be uploaded. Log into the Bintray UI and create a package for <code>cloudfoundry-stub-runner-boot</code>. If you forked this repo, you can use the <code>Import from GitHub</code> option. Otherwise, create the package manually and choose any license (for example, Apache 2.0).</p>
</div>
<div class="paragraph">
<p>Now you are ready to build and upload this app to Bintray. Edit the following script (which shows cloning, building, and uploading) to match your Bintray URL, the Bintray ID in your <code>~/.m2/settings/xml</code> file, and the <code>cloudfoundry-stub-runner-boot</code> repository URL (if you chose to fork it):</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash"># Edit to match your Bintray URL and M2 repo ID setting (check your ~/.m2/settings.xml file)
MAVEN_REPO_URL=https://api.bintray.com/maven/ciberkleid/maven-repo/cloudfoundry-stub-runner-boot
MAVEN_REPO_ID=bintray

# Clone cloudfoundry-stub-runner-boot
git clone https://github.com/spring-cloud-samples/cloudfoundry-stub-runner-boot.git
cd cloudfoundry-stub-runner-boot

# Build and upload
./mvnw clean deploy -Ddistribution.management.release.url="${MAVEN_REPO_URL}" -Ddistribution.management.release.id="${MAVEN_REPO_ID}"</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>You should now see the <code>cloudfoundry-stub-runner-boot</code> artifacts uploaded on Bintray.</p>
</div>
</div>
<div class="sect5">
<h6 id="provide-stand-alone-stub-runner-application-manifest"><a class="link" href="#provide-stand-alone-stub-runner-application-manifest">Provide Stand-alone Stub Runner Application Manifest</a></h6>
<div class="paragraph">
<p>Next, we add a manifest file for the stub runner application for deployment to Cloud Foundry.</p>
</div>
<div class="paragraph">
<p>Place this file in the <code>greeting-ui</code> repo. The file name and location can be your choice. For this example, we use <code>cloud-pipelines/manifest-stubrunner.yml</code>. The following image shows the file in the appropriate folder:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/greeting_ui_stubrunner_manifest.png" alt="greeting ui stubrunner manifest">
</div>
<div class="title">Figure 21. greeting-ui Stub Runner Manifest</div>
</div>
<div class="paragraph">
<p>We populate this <code>manifest-stubrunner.yml</code> with the content shown in the next listing so that the stub runner binds to <code>service-registry</code>. The stub runner registers the <code>fortune-service</code> stub there so that <code>greeting-ui</code> can discover it in the same way it discovers the actual <code>fortune-service</code> app endpoint in stage and prod. From the <code>greeting-ui</code> perspective, there is no difference in how it interacts with Eureka and the stub runner application in test and the way it interacts with Eureka and the <code>fortune-service</code> application in stage and prod. The following listing shows the content of <code>manifest-stubrunner.yml</code>:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">---
applications:
- name: stubrunner
  timeout: 120
  services:
  - service-registry
  env:
    JAVA_OPTS: -Djava.security.egd=file:///dev/urandom
    TRUST_CERTS: api.run.pivotal.io
----</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect5">
<h6 id="provide-a-stub-runner-jar-and-manifest-information-to-the-pipeline"><a class="link" href="#provide-a-stub-runner-jar-and-manifest-information-to-the-pipeline">Provide a Stub Runner Jar and Manifest Information to the Pipeline</a></h6>
<div class="paragraph">
<p>Now that we have a jar file and a manifest file for our stub runner application, we need to provide this information to our <code>greeting-ui</code> pipeline so that the pipeline downloads the jar from Bintray and deploys it to Cloud Foundry. We do this through the <code>greeting-ui</code> <code>cloud-pipelines.yml</code> file. We add an entry to the list of services in the <code>test</code> section, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-yml hljs" data-lang="yml">    - name: stubrunner
      type: stubrunner
      coordinates: io.pivotal:cloudfoundry-stub-runner-boot:0.0.1.M1
      pathToManifest: cloud-pipelines/manifest-stubrunner.yml</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Notice that <code>name</code> matches the name of the application in <code>manifest-stubrunner.yml</code>, <code>coordinates</code> corresponds to the jar coordinates of the Maven repository, and <code>pathToManifest</code> matches our chosen fie name for the stub runner application manifest.</p>
</div>
<div class="paragraph">
<p>Note also the <code>type</code> is set to <code>stubrunner</code>, which Cloud Pipelines will recognize as a stanalone stub runner app and treat accordingly.</p>
</div>
</div>
<div class="sect5">
<h6 id="provide-stub-configuration-for-the-stub-runner-application"><a class="link" href="#provide-stub-configuration-for-the-stub-runner-application">Provide Stub Configuration for the Stub Runner Application</a></h6>
<div class="paragraph">
<p>The final steps in the configuration of the stand-alone stub runner app are as follows:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Enable the stub runner app to download the <code>fortune-service</code> stub from Bintray.</p>
</li>
<li>
<p>Open a second port on the container to receive requests for this stub.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>To accomplish this, we put stub and port configuration information into the properties section of the <code>greeting-ui</code> <code>pom.xml</code> file, by using a property called <code>stubrunner.ids</code>. This property can accept a list of stubrunner IDs. However, for this tutorial, we only have one, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-xml hljs" data-lang="xml">  &lt;properties&gt;
...
    &lt;!--Tell stub runner app to start this stub--&gt;
    &lt;stubrunner.ids&gt;io.pivotal:fortune-service:1.0.0.M1-20180102_203542-VERSION:stubs:10000&lt;/stubrunner.ids&gt;
  &lt;/properties&gt;</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>Cloud Pipelines uses this information in two ways:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>It provides this information to the stub runner application through the application&#8217;s environment variables.</p>
<div class="ulist">
<ul>
<li>
<p>Cloud Pipelines also provides the <code>$REPO_WITH_BINARIES</code> as an environment variable for the stub runner application.</p>
</li>
<li>
<p>The stub runner application uses this information to download the stub from Bintray and expose it on the specified port.</p>
</li>
</ul>
</div>
</li>
<li>
<p>It opens the additional port specified on the stub runner app and map a new route to it.</p>
<div class="ulist">
<ul>
<li>
<p>The format for each route is <code>&lt;stub-runner-app-name&gt;-&lt;hostname-uuid&gt;-&lt;env&gt;-&lt;app-name&gt;-&lt;port&gt;.&lt;domain&gt;</code>.</p>
</li>
<li>
<p>In our example, this would be <code>stubrunner-cyi-test-greeting-ui-10000.cfapps.io</code>.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
<div class="paragraph">
<p>Since we bound our stub runner application to <code>service-registry</code> (Eureka), the stub runner application registers the stub URL under the <code>FORTUNE-SERVICE</code> application name on Eureka, as the following image shows:</p>
</div>
<div class="imageblock">
<div class="content">
<img src="https://raw.githubusercontent.com/CloudPipelines/concourse/master/docs/images/cf-migration/greeting_ui_stub_runner_eureka_registration.png" alt="greeting ui stub runner eureka registration">
</div>
<div class="title">Figure 22. greeting-ui Stub Runner Eureka Registration</div>
</div>
<div class="paragraph">
<p>This completes the process of configuring the stand-alone stub runner application.</p>
</div>
<div class="admonitionblock note">
<table>
<tr>
<td class="icon">
<i class="fa icon-note" title="Note"></i>
</td>
<td class="content">
You can automate the port configuration by Cloud Pipelines in the future such that you need not include the port in <code>stubrunner.ids</code>. However, for the moment, we are required to specify the port each stub should use.
</td>
</tr>
</table>
</div>
</div>
<div class="sect5">
<h6 id="edit-smoke-tests-to-align-with-the-contract"><a class="link" href="#edit-smoke-tests-to-align-with-the-contract">Edit Smoke Tests to Align with the Contract</a></h6>
<div class="paragraph">
<p>Finally, we edit our smoke tests for <code>greeting-ui</code> to ensure the response does not contain the Hystrix fallback, since we are now expecting a response from the stub. The following listing shows the edited smoke tests:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-java hljs" data-lang="java">package smoke;

import org.assertj.core.api.BDDAssertions;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.http.ResponseEntity;
import org.springframework.test.context.junit4.SpringRunner;
import org.springframework.web.client.RestTemplate;

@RunWith(SpringRunner.class)
@SpringBootTest(classes = SmokeTests.class,
        webEnvironment = SpringBootTest.WebEnvironment.NONE)
@EnableAutoConfiguration
public class SmokeTests {

	@Value("${application.url}") String applicationUrl;

	RestTemplate restTemplate = new RestTemplate();

	@Test
	public void should_return_a_fortune() {
		ResponseEntity&lt;String&gt; response = this.restTemplate
				.getForEntity("http://" + this.applicationUrl + "/", String.class);

		BDDAssertions.then(response.getStatusCodeValue()).isEqualTo(200);

		// Filter out the known Hystrix fallback response
		BDDAssertions.then(response.getBody()).doesNotContain("This fortune is no good. Try another.");
	}

}</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>In this case, in contrast to the integration test we created earlier for <code>greeting-ui</code>, we do not include <code>@AutoConfigureStubRunner</code>, because we are using a standalone stub runner application.</p>
</div>
</div>
</div>
<div class="sect4">
<h5 id="3-9-push-changes-to-github"><a class="link" href="#3-9-push-changes-to-github">3.9 Push Changes to GitHub</a></h5>
<div class="paragraph">
<p>Push contract-based changes for <code>greeting-ui</code>. You should be pushing the following new or modified files:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><code>pom.xml</code></p>
</li>
<li>
<p><code>cloud-pipelines.yml</code></p>
</li>
<li>
<p><code>cloud-pipelines/manifest-stubrunner.yml</code></p>
</li>
<li>
<p><code>src/test/java/io/pivotal/fortune/FortuneServiceTests.java</code></p>
</li>
<li>
<p><code>src/test/java/smoke/SmokeTests.java</code></p>
</li>
</ul>
</div>
<div class="paragraph">
<p>At this point, we can run through the full pipeline for <code>greeting-ui</code> and leverage the contract-based stub in both the build and test environments.</p>
</div>
</div>
<div class="sect4">
<h5 id="stage-three-recap"><a class="link" href="#stage-three-recap">Stage Three Recap</a></h5>
<div class="paragraph">
<p>What have we accomplished?</p>
</div>
<div class="paragraph">
<p>By implementing a contract-driven approach with auto-generation of tests and stubs, we have introduced a clean, structured, and reliable way to define, communicate, document, manage, and test APIs. The benefits include the following:</p>
</div>
<div class="ulist">
<ul>
<li>
<p>Inter-team communication can be simpler.</p>
<div class="ulist">
<ul>
<li>
<p>Consumer and producer teams can now communicate requirements through coded contracts.</p>
</li>
<li>
<p>The inventory of contracts serves as a record and reference of the agreed-upon APIs.</p>
</li>
</ul>
</div>
</li>
<li>
<p>Developer productivity will increase.</p>
<div class="ulist">
<ul>
<li>
<p>Producers can quickly and easily generate contract-based stubs.</p>
</li>
<li>
<p>Consumers no longer have to manually stub out APIs and write tests with arbitrary hard-coded responses. Instead, they can use the auto-generated stubs and test for contract-based responses.</p>
</li>
<li>
<p>Both producers and consumers can validate their code complies with the contract.</p>
</li>
<li>
<p>Producers can verify backward compatibility of API changes.</p>
</li>
<li>
<p>Troubleshooting will be easier.</p>
</li>
<li>
<p>Failure and feedback will be faster.</p>
</li>
</ul>
</div>
</li>
</ul>
</div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="conclusion"><a class="link" href="#conclusion">Conclusion</a></h3>
<div class="paragraph">
<p>This concludes the tutorial on migrating apps for Cloud Pipelines for Cloud Foundry.</p>
</div>
<div class="paragraph">
<p>Moving forward, the refactoring work shown here can be incorporated into your and your team&#8217;s standard practices. We recommend the following practices:</p>
</div>
<div class="paragraph">
<p><strong>Good:</strong></p>
</div>
<div class="ulist">
<ul>
<li>
<p>Use Maven or Gradle wrappers.</p>
</li>
<li>
<p>Include a Cloud Foundry manifest file in your application repository.</p>
</li>
<li>
<p>Include a pipeline descriptor (<code>sc-manifest.yml</code>) in your app repository.</p>
</li>
<li>
<p>Create an empty <code>version</code> branch in your application repository.</p>
</li>
<li>
<p>Include artifact repository configuration in the <code>pom.xml</code> file.</p>
</li>
<li>
<p>Align your Cloud Foundry spaces with the Cloud Pipelines model (isolated test space and shared stage and prod spaces).</p>
</li>
</ul>
</div>
<div class="paragraph">
<p><strong>Better</strong></p>
</div>
<div class="ulist">
<ul>
<li>
<p>Include <code>default</code>, <code>apicompatibility</code>, <code>smoke</code>, and <code>e2e</code> profiles in the <code>pom.xml</code> file.</p>
</li>
<li>
<p>Organize tests accordingly in your application repository.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p><strong>Best</strong></p>
</div>
<div class="ulist">
<ul>
<li>
<p>Use a database migration tool, such as Flyway.</p>
</li>
<li>
<p>Use contract-based API programming.</p>
</li>
</ul>
</div>
<div class="paragraph">
<p>Implementing all the <strong>&#8220;good&#8221;</strong> practices positions you to instantly create pipelines for your applications by using Cloud Pipelines. This is a huge win in terms of consistency and productivity and standardization across development teams. Of course, this is an open source project, so you can modify it to meet your needs.</p>
</div>
<div class="paragraph">
<p>Implementing the <strong>&#8220;better&#8221;</strong> practices ensures the proper tests get run at the proper time. At that point, you can add as much test coverage as you need to have high confidence in your pipelines.</p>
</div>
<div class="paragraph">
<p>Implementing the <strong>&#8220;best&#8221;</strong> practices gives you additional confidence in your pipeline and encourages better programming practices for database version and API management across development teams. It also gives you higher confidence in your pipelines and lets you avoid the cumbersome business of rolling back a database.</p>
</div>
<div class="paragraph">
<p>Happy coding!</p>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="setup-examples"><a class="link" href="#setup-examples">Setup examples</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>You can go to <a href="SETUP.html">setup subpage</a> to read more
about different setup scenarios.</p>
</div>
</div>
</div>
<div class="sect1">
<h2 id="building-the-project"><a class="link" href="#building-the-project">Building the Project</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>This section covers how to build the project. It covers:</p>
</div>
<div class="ulist">
<ul>
<li>
<p><a href="#building-project-setup">Project Setup</a></p>
</li>
<li>
<p><a href="#building-prerequisites">Prerequisites</a></p>
</li>
<li>
<p><a href="#building-bats-submodules">Bats Submodules</a></p>
</li>
<li>
<p><a href="#building-build-and-test">Build and test</a></p>
</li>
<li>
<p><a href="#building-generate-documentation">Generate Documentation</a></p>
</li>
<li>
<p><a href="#building-ci-worker-prerequisites">CI Server Worker Prerequisites</a></p>
</li>
</ul>
</div>
<div class="sect2">
<h3 id="building-project-setup"><a class="link" href="#building-project-setup">Project Setup</a></h3>
<div class="paragraph">
<p>The project contains the following structure.</p>
</div>
<div class="paragraph">
<p>Under <code>demo</code> folder you can find the setup prepared to start a demo instance of Concourse and Artifactory.</p>
</div>
<div class="paragraph">
<p>In <code>docs</code> you can find HTML documentation built from contents of <code>docs-sources</code>.</p>
</div>
<div class="paragraph">
<p>The <code>src/pipeline</code> folder contains the files related to setting up pipelines in Concourse.
In <code>src/tasks</code> we&#8217;ll find all the scripts required at runtime of pipeline execution.</p>
</div>
<div class="paragraph">
<p>The <code>tools</code> folder contains tools required for building and that ease the work
with demo setup.</p>
</div>
</div>
<div class="sect2">
<h3 id="building-prerequisites"><a class="link" href="#building-prerequisites">Prerequisites</a></h3>
<div class="paragraph">
<p>As prerequisites, you need to have <a href="https://www.shellcheck.net/">shellcheck</a>,
<a href="https://github.com/sstephenson/bats">bats</a>, <a href="https://stedolan.github.io/jq/">jq</a>
and <a href="https://rubyinstaller.org/downloads/">ruby</a> installed. If you use a Linux
machine, <code>bats</code> and <code>shellcheck</code> are installed for you.</p>
</div>
<div class="paragraph">
<p>To install the required software on Linux, type the following command:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ sudo apt-get install -y ruby jq</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>If you use a Mac, run the following commands to install the missing software:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ brew install jq
$ brew install ruby
$ brew install bats
$ brew install shellcheck</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="building-bats-submodules"><a class="link" href="#building-bats-submodules">Bats Submodules</a></h3>
<div class="paragraph">
<p>To make <code>bats</code> work properly, we needed to attach Git submodules. To have them
initialized, either clone the project or (if you have already cloned the project)
pull to update it. The following command clones the project:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ git clone --recursive https://github.com/cloudpipelines/concourse.git</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>The following commands pull the project:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ git submodule init
$ git submodule update</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>If you forget about this step, Gradle runs these steps for you.</p>
</div>
</div>
<div class="sect2">
<h3 id="building-build-and-test"><a class="link" href="#building-build-and-test">Build and test</a></h3>
<div class="paragraph">
<p>Once you have installed all the prerequisites, you can run the following command to build and test the project:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ ./gradlew clean build</code></pre>
</div>
</div>
</div>
</div>
</div>
<div class="sect2">
<h3 id="building-generate-documentation"><a class="link" href="#building-generate-documentation">Generate Documentation</a></h3>
<div class="paragraph">
<p>To generate the documentation, run the following command:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">$ ./gradlew generateDocs</code></pre>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="sect1">
<h2 id="building-ci-worker-prerequisites"><a class="link" href="#building-ci-worker-prerequisites">CI Server Worker Prerequisites</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>Cloud Pipelines uses Bash scripts extensively. The following list shows the software
that needs to be installed on a CI server worker for the build to pass:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash"> apt-get -y install \
    bash \
    git \
    tar \
    zip \
    curl \
    ruby \
    wget \
    unzip \
    python \
    jq</code></pre>
</div>
</div>
</div>
</div>
<div class="admonitionblock tip">
<table>
<tr>
<td class="icon">
<i class="fa icon-tip" title="Tip"></i>
</td>
<td class="content">
In the demo setup all of these libraries are already installed.
</td>
</tr>
</table>
</div>
</div>
</div>
<div class="sect1">
<h2 id="concourse-faq"><a class="link" href="#concourse-faq">Concourse FAQ</a></h2>
<div class="sectionbody">
<div class="paragraph">
<p>This section covers the most commonly asked questions about using Concourse with Cloud Pipelines:</p>
</div>
<div class="dlist">
<dl>
<dt class="hdlist1">Can I use the pipeline for some other repos?</dt>
<dd>
<p>Yes To do so, change the <code>app-url</code> in <code>credentials.yml</code>!</p>
</dd>
<dt class="hdlist1">Does this work for ANY project out of the box?</dt>
<dd>
<p>Not really. This is an <code>opinionated pipeline</code>. That is why we took some
opinionated decisions. See the documentation to learn
what those decisions are.</p>
</dd>
<dt class="hdlist1">I ran out of resources! (PCF Dev)</dt>
<dd>
<p><a id="resources"></a> When deploying the application to stage or prod, you can get an <code>Insufficient resources</code> exception. The way to
resolve it is to kill some apps from the test or stage environment. To do so, run the following commands:</p>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">cf target -o pcfdev-org -s pcfdev-test
cf stop github-webhook
cf stop github-eureka
cf stop stubrunner</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>You can also run <code>./tools/cf-helper.sh kill-all-apps</code> to remove
all demo-related apps deployed to PCF Dev.</p>
</div>
</dd>
<dt class="hdlist1">The rollback step fails due to a missing JAR.</dt>
<dd>
<p>You must have pushed some tags and must have also removed the Artifactory volume that
contained them. To fix this, remove the tags by running the following command:</p>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">git tag -l | xargs -n 1 git push --delete origin</code></pre>
</div>
</div>
</div>
</div>
</dd>
<dt class="hdlist1">Can I see the output of a job from the terminal?</dt>
<dd>
<p>Yes. Assuming that pipeline name is <code>github-webhook</code> and the job name is
<code>build-and-upload</code> you can see the output by running the following command:</p>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">fly watch --job github-webhook/build-and-upload -t docker</code></pre>
</div>
</div>
</div>
</div>
</dd>
<dt class="hdlist1">I clicked the job and it is constantly pending.</dt>
<dd>
<p>Most likely, you forgot to click the <code>play</code> button to
unpause the pipeline. Click the top left, expand the list of pipelines, and click
the <code>play</code> button next to <code>github-webhook</code>.</p>
<div class="paragraph">
<p>Another problem that might occur is that you need to have the <code>version</code> branch.
Concourse waits for the <code>version</code> branch to appear in your repository. So, for
the pipeline to start, ensure that when doing some git operations, you have not
forgotten to create and copy the <code>version</code> branch too.</p>
</div>
</dd>
<dt class="hdlist1">The route is already in use (CF)</dt>
<dd>
<p>If you play around with Concourse you might end up with the routes occupied,
as indicated by a message similar to the following:</p>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">Using route github-webhook-test.local.pcfdev.io
Binding github-webhook-test.local.pcfdev.io to github-webhook...
FAILED
The route github-webhook-test.local.pcfdev.io is already in use.</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>To fix the problem, you can delete the routes, as follows:</p>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">yes | cf delete-route local.pcfdev.io -n github-webhook-test
yes | cf delete-route local.pcfdev.io -n github-eureka-test
yes | cf delete-route local.pcfdev.io -n stubrunner-test
yes | cf delete-route local.pcfdev.io -n github-webhook-stage
yes | cf delete-route local.pcfdev.io -n github-eureka-stage
yes | cf delete-route local.pcfdev.io -n github-webhook-prod
yes | cf delete-route local.pcfdev.io -n github-eureka-prod</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>You can also run the <code>./tools/cf-helper.sh delete-routes</code> script.</p>
</div>
</dd>
<dt class="hdlist1">I mm unauthorized to deploy infrastructure jars</dt>
<dd>
<p>Most likely, you forgot to update your local <code>settings.xml</code> file with Artifactory&#8217;s
setup. See <a href="#settings">this section of the docs and update your <code>settings.xml</code> file</a>.</p>
</dd>
<dt class="hdlist1">The <code>version</code> resource is broken. When I click on it, I get the following error</dt>
</dl>
</div>
<div class="exampleblock">
<div class="content">
<div class="listingblock">
<div class="content">
<pre class="highlightjs highlight"><code class="language-bash hljs" data-lang="bash">resource script '/opt/resource/check []' failed: exit status 128

stderr:
Identity added: /tmp/git-resource-private-key (/tmp/git-resource-private-key)
Cloning into '/tmp/git-resource-repo-cache'...
warning: Could not find remote branch version to clone.
fatal: Remote branch version not found in upstream origin</code></pre>
</div>
</div>
</div>
</div>
<div class="paragraph">
<p>That means that your repo does not have the <code>version</code> branch. You need
set it up.</p>
</div>
</div>
</div>
</div>
<link rel="stylesheet" href="js/highlight/styles/atom-one-dark-reasonable.min.css">
<script src="js/highlight/highlight.min.js"></script>
<script>hljs.initHighlighting()</script>
</body>
</html>